Welding information acquisition device

Abstract

This welding information acquisition device comprises an arithmetic unit that performs the following: scanning, using a three-dimensional shape measurement sensor, the periphery of a weld bead that joins two base materials, to obtain three-dimensional shape data; with regard to the three-dimensional shape data, identifying, when the coordinate in a predetermined direction of each measurement point is regarded as a function of the coordinate in a first direction which is perpendicular to the predetermined direction, the maximum change point at which the derivative of the function becomes the greatest for each point group sharing a coordinate in a second direction which is perpendicular to the predetermined direction and to the first direction which is perpendicular to the predetermined direction; and estimating a boundary line FL1 of the two base materials on the basis of the coordinates of, among the maximum change points, a plurality of maximum change points present outside the weld bead formation region.

Description

Welding information acquisition device

[0001] The present disclosure relates to a welding information acquisition device that acquires information about a welding state based on three-dimensional shape data.

[0002] Patent Document 1 discloses a welding information acquisition device that acquires information about the quality of the shape of a welded portion based on three-dimensional shape data and a judgment model.

[0003] International Publication No. 2020 / 129617

[0004] Incidentally, there is a demand for obtaining information regarding the positions of the two base materials to be welded in order to determine whether the shape of the weld is good or bad, as disclosed in Patent Document 1, and to calculate various parameters such as the positional deviation of the weld line, leg length, throat depth, etc.

[0005] The present disclosure has been made in consideration of the above points, and its purpose is to obtain information regarding the positions of two base materials to be welded.

[0006] In order to achieve the above-mentioned object, a first aspect of the present disclosure is characterized by comprising: a shape data storage unit that stores three-dimensional shape data obtained by scanning around a weld bead joining two base materials with a sensor, and that indicates the coordinates of multiple measurement points on the surface of the weld bead and the exposed surface of the base material; and a calculation unit that generates bead area data based on the shape data to identify the formation area of ​​the weld bead in a plane perpendicular to the direction in which the sensor faces; identifies, for each point group having a common coordinate in a second direction perpendicular to the direction in which the sensor faces and the first direction, a maximum change point at which the differential coefficient of the function is maximum when the coordinate of each measurement point in the direction in which the sensor faces is considered as a function of the coordinate in the first direction perpendicular to the direction in which the sensor faces; and estimates the boundary line of the two base materials based on the coordinates of multiple maximum change points that are outside the formation area of ​​the weld bead identified by the bead area data.

[0007] This allows the position of the boundary line between the two base materials to be estimated.

[0008] A second aspect of the present disclosure includes an image data storage unit that stores three-dimensional shape data that is acquired by scanning the periphery of a weld bead (a region including the weld bead) that joins two base materials with a sensor, and indicates the coordinates of a plurality of measurement points on the surface of the weld bead and on the exposed surface of the base material, and stores image data as a point cloud in which the plurality of measurement points are arranged at a first constant interval in the X-axis direction of the XYZ orthogonal coordinate system and at a second constant interval in the Y-axis direction of the XYZ orthogonal coordinate system, in an XYZ orthogonal coordinate system in which the Z-axis direction is the direction in which the sensor faces; The present invention is characterized in that it comprises a calculation unit that generates bead area data that specifies a formation area of ​​the weld bead in a plane perpendicular to the direction in which the sensor faces, based on the shape data, and estimates a base material line that indicates the exposed surfaces (base material surfaces) of the two base materials in a cross section at the width line, based on the Z coordinate of a measurement point, among constituent points on a width line extending in the width direction of the weld bead, that is outside the formation area of ​​the weld bead specified by the bead area data, has the same X coordinate as a constituent point that is not the measurement point, and is closest in the Y-axis direction to the constituent point.

[0009] This allows the positions of the exposed surfaces of the two base materials to be estimated.

[0010] According to the present disclosure, information regarding the positions of two base materials to be welded can be obtained.

[0011] FIG. 1 is a functional block diagram of a welding information acquisition device according to an embodiment. FIG. 2 is a schematic diagram of the hardware configuration of the welding information acquisition device. FIG. 3 is a schematic diagram showing how shape data of a weld bead is acquired by a visual inspection device. FIG. 4 is a flowchart showing a procedure for measuring dimensions of a weld bead. FIG. 5 is a schematic diagram showing the outline of a weld bead during smoothing processing and when acquiring endpoints. FIG. 6 is a flowchart showing a procedure for extracting a weld bead formation area. FIG. 7 is a diagram showing an image of shape data and a weld bead formation area and other areas in the shape data. FIG. 8 is a flowchart showing a procedure for adding annotations for generating learning data. FIG. 9A is an example showing how annotations are added to shape data. FIG. 9B is an example showing an image of the shape data after annotations are added. FIG. 10 is a flowchart showing a procedure for adding first and second reference lines. FIG. 11A is a schematic diagram showing a first loop process when adding first and second reference lines. FIG. 11B is a schematic diagram showing a process subsequent to the process shown in FIG. 11A. FIG. 11C is a schematic diagram showing processing subsequent to the processing shown in FIG. 11B. FIG. 12 is a schematic diagram showing second loop processing when adding a first reference line and a second reference line. FIG. 13 is a flowchart showing a procedure for correcting the first reference line. FIG. 14A is an example of contour data to which the first reference line is added when no correction is performed. FIG. 14B is an example of another example of contour data to which the first reference line is added when no correction is performed. FIG. 15A is an example of contour data to which the first reference line is transformed into a provisional reference line. FIG. 15B is an example of another example of contour data to which the first reference line is transformed into a provisional reference line. FIG. 16A is an example of contour data to which the first reference line is added after correction is completed. FIG. 16B is an example of another example of contour data to which the first reference line is added after correction is completed. FIG. 17 is a flowchart showing a procedure for measuring a positional deviation of a weld bead. FIG. 18 is an explanatory diagram explaining a procedure for estimating a boundary line between two base materials. Fig. 19 is a graph showing the height of the base metal surface and the differential coefficient (differential value) along line A-A in Fig. 18. Fig. 20 is a flowchart showing the procedure for identifying the longitudinal and width directions of the weld bead. Fig. 21 is an explanatory diagram for explaining a method of converting shape data.FIG. 22 is a graph illustrating the distribution of measurement points constituting the first reference line. FIG. 23 is a flowchart showing the procedure for measuring the leg length and throat depth of a weld bead. FIG. 24 is an explanatory diagram illustrating a width line. FIG. 25 is an enlarged view of a portion of the width line in FIG. 24 . FIG. 26 is a table illustrating the coordinates of the constituent points constituting the width line. FIG. 27 is a cross-sectional view at the width line when two base materials form a T-joint. FIG. 28 is an enlarged view of the boundary of the weld bead in FIG. 27 showing the base material line before and after translation. FIG. 29 is a diagram equivalent to FIG. 27 when two base materials form a lap joint. FIG. 30 is a table illustrating first and second leg lengths and throat depths acquired visually and by a welding information acquisition device, as well as the difference between their numerical values. FIG. 31 is a diagram equivalent to FIG. 29 showing the first and second leg lengths and throat depths based on the boundary line of the base material.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. The following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or uses in any way.

[0013] [1: Configuration of Weld Bead Welding Information Acquisition Device] FIG. 1 shows a functional block diagram of a welding information acquisition device according to this embodiment, and FIG. 2 shows a schematic diagram of the hardware configuration of the welding information acquisition device.

[0014] 1 , welding information acquisition device 20 includes a data acquisition unit 1, a bead region extraction unit 2, a contour extraction unit 3, a smoothing processing unit 4, an end point acquisition unit 5, a reference line assignment unit 6, and a dimension calculation unit 7. Welding information acquisition device 20 also includes a storage unit 8 serving as a shape data storage unit and / or an image data storage unit, a display unit 9, and a learning data generation unit 10. Welding information acquisition device 20 also includes a maximum change point identification unit 11, a boundary line estimation unit 12, a positional deviation amount calculation unit 13, a base wire estimation unit 14, a joint determination unit 15, and a leg length / throat thickness calculation unit 16. The calculation unit 28 is composed of a data acquisition unit 1, a bead region extraction unit 2, a contour extraction unit 3, a smoothing processing unit 4, an end point acquisition unit 5, a reference line assignment unit 6, a dimension calculation unit 7, a learning data generation unit 10, a maximum change point identification unit 11, a boundary line estimation unit 12, a positional deviation amount calculation unit 13, a base wire estimation unit 14, a joint determination unit 15, and a leg length / throat thickness calculation unit 16.

[0015] The data acquiring unit 1 acquires three-dimensional shape data (hereinafter simply referred to as shape data) from the appearance inspection device 30 (see FIG. 3). Note that, although FIG. 1 shows an example in which the shape data is directly input from the appearance inspection device 30 to the data acquiring unit 1, this is not particularly limited. For example, the shape data may be stored in a storage unit (not shown) provided outside the welding information acquisition device 20, and the data acquiring unit 1 may acquire the shape data from the storage unit.

[0016] 3, the visual inspection device 30 is configured by attaching a three-dimensional shape measurement sensor 31 to a robot 32. The three-dimensional shape measurement sensor 31 acquires shape data by scanning the periphery of a weld bead 110 that joins two base materials 100. The three-dimensional shape measurement sensor 31 may be attached to a welding head (not shown) held by the robot 32. The welding head is an energy supply unit that supplies energy for welding to the base materials 100, and refers to, for example, a welding torch or a laser head.

[0017] The three-dimensional shape measuring sensor 31 is composed of a laser light source (not shown) configured to be able to scan the surface of the base material 100, and a camera or a light receiving sensor (neither of which are shown) that captures an image of the reflection trajectory of the laser beam projected onto the surface of the base material 100. The camera has a CCD or CMOS image sensor as an imaging element. The configuration of the three-dimensional shape measuring sensor 31 is not limited to the above, and other configurations may be used. For example, an optical interferometer may be used instead of the camera or light receiving sensor.

[0018] The three-dimensional shape measurement sensor 31 scans the weld bead 110 and the surrounding base material 100 with a laser beam, and the laser beam reflected by the weld bead 110 and the base material 100 is captured by a camera or a light-receiving sensor, thereby obtaining the three-dimensional shape of the weld bead 110 and the surrounding base material 100, i.e., shape data.

[0019] The shape data acquired by the three-dimensional shape measurement sensor 31 may include not only position information but also color information. Furthermore, when the imaging element or light receiving sensor of the three-dimensional shape measurement sensor 31 acquires a color image or a black-and-white image, each pixel includes position information and color information. In other words, the data of each pixel that constitutes the shape data, i.e., the pixel data, includes position information and color information.

[0020] Here, "position information" refers to three-dimensional coordinate information with a predetermined position on the surface of the base material 100 as the origin. Here, the direction in which the three-dimensional shape measurement sensor 31 faces is defined as the Z-axis direction. In other words, the direction in which the three-dimensional shape measurement sensor 31 faces is defined as the Z-axis direction, which is perpendicular to the surface of the base material 100, so that the sensor faces the surface of the base material 100. The plane perpendicular to the Z-axis direction is defined as the XY plane. The three-dimensional shape measurement sensor 31 is a line sensor with multiple image sensors arranged at 0.025 mm intervals in a first direction. The three-dimensional shape measurement sensor 31 captures images every 0.1 mm of movement while moving in a second direction perpendicular to the first direction. Here, the first direction (the direction in which the image sensors of the three-dimensional shape measurement sensor 31 are aligned) is defined as the X-axis direction, and the second direction (the direction in which the three-dimensional shape measurement sensor 31 moves) is defined as the Y-axis direction. Specifically, the position information indicates the coordinates (X, Y, Z) of each measurement point on the surface of weld bead 110 and the exposed surface of base material 100. The Z coordinate indicates the distance between three-dimensional shape measurement sensor 31 and the measurement point on the surface of weld bead 110 and the exposed surface of base material 100.

[0021] Therefore, the shape data indicates the coordinates of a plurality of measurement points in an XYZ Cartesian coordinate system in which the direction in which the three-dimensional shape measuring sensor 31 faces is the Z-axis direction.

[0022] The direction in which the three-dimensional shape measurement sensor 31 faces the base material 100 may not only be perpendicular to the surface of the base material 100 (a predetermined direction) but also be tilted at a predetermined angle from the direction perpendicular to the surface of the base material 100 (or may be tilted from the initial angle of the direction in which the three-dimensional shape measurement sensor 31 faces). In this case, the predetermined angle (irradiation angle) is set in a setting file in advance. The bead area extraction unit 2 then corrects the shape data according to the predetermined angle, stores the corrected shape data in the storage unit 8, and generates bead area data based on the corrected shape data. Specifically, in the welding information acquisition device 20, the direction in which the three-dimensional shape measurement sensor 31 faces is tilted by a predetermined angle from the predetermined direction in which the three-dimensional shape measurement sensor 31 faces, and the three-dimensional shape measurement sensor 31 scans around the weld bead 110 to acquire shape data. The bead area extraction unit 2 then corrects the shape data according to the predetermined tilt angle. This makes it possible to obtain the correct leg length, throat thickness, positional deviation, etc., and improves the degree of freedom in the placement and setting of the three-dimensional shape measurement sensor 31.

[0023] The plurality of measurement points are arranged at regular intervals of 0.025 mm (first intervals) in the X-axis direction of the XYZ Cartesian coordinate system and at regular intervals of 0.1 mm (second intervals) in the Y-axis direction of the XYZ Cartesian coordinate system. The intervals between the measurement points in the X-axis direction may be a length other than 0.025 mm. The intervals between the measurement points in the Y-axis direction may be a length other than 0.1 mm. For ease of understanding, the grid coordinates may be configured such that the regular intervals between the measurement points in the XYZ Cartesian coordinate system in the X-axis direction (first intervals) are 1 (one square), and the regular intervals between the measurement points in the Y-axis direction (second intervals) are 1 (one square). In this case, the regular intervals between the measurement points in the XYZ Cartesian coordinate system in the X-axis direction (0.025 mm, first intervals) are one square, and the regular intervals between the measurement points in the XYZ Cartesian coordinate system in the Y-axis direction (0.1 mm, second intervals) are one square. Specifically, the shape data indicates the coordinates of multiple measurement points such that the X coordinates of adjacent measurement points in the X-axis direction differ by one (one square) and the Y coordinates of adjacent measurement points in the Y-axis direction differ by one (one square). Note that the data format in which the X coordinates of adjacent measurement points in the X-axis direction differ by one (one square) and the Y coordinates of adjacent measurement points in the Y-axis direction differ by one (one square) is merely an example, and other data formats may be used for the shape data. Furthermore, "color information" refers to information about color components included in the signal output from the three-dimensional shape measurement sensor 31. When the three-dimensional shape measurement sensor 31 acquires a color image, the color information included in the pixel data may be, for example, luminance information for each of the R component (red component), G component (green component), and B component (blue component). When the three-dimensional shape measurement sensor 31 acquires a black-and-white image, the color information included in the pixel data is luminance information. Furthermore, the color information may include information corresponding to the luminance of the component (A component) representing transparency. The magnitude of luminance may be normalized, for example.

[0024] Furthermore, the "color information" may be information about the type of color. For example, if a table in which a number is assigned to each type of color is prepared, the color information corresponds to the number corresponding to the type of color.

[0025] In either case, shape data is constructed with position information combined with color information at that position.

[0026] The bead area extraction unit 2 generates bead area data that identifies the formation area of ​​the weld bead 110 in a plane perpendicular to the direction in which the three-dimensional shape measurement sensor 31 faces, i.e., the XY plane, based on the shape data stored in the memory unit 8 (described later). In other words, the bead area extraction unit 2 extracts the formation area of ​​the weld bead 110 from the shape data as bead area data in a two-dimensional data format that does not include height information. This allows for the extraction of bead area data for extracting the length and width of the weld bead in a simple two-dimensional data format. Because machine learning (deep learning) is used to extract the bead area data, various settings (subtle parameter settings for various reference values) are not required for measuring the weld bead length and width, as in conventional methods, depending on the welding method (arc welding, laser welding, etc.), weld joint (lap, butt, etc.), weld bead shape (straight, curved, etc.), etc. This allows for easy extraction of the weld bead area, making the operation very familiar even for new users. The method for extracting bead area data will be described later.

[0027] Based on the bead region data, contour extraction unit 3 extracts the contour of weld bead 110 as contour data. The method of extracting the contour data will be described later.

[0028] The smoothing processor 4 performs smoothing processing on the contour data.

[0029] End point acquisition unit 5 acquires end points (hereinafter sometimes simply referred to as end points) of weld bead 110 based on the smoothed contour data. Note that, as will be described later, multiple end points may be acquired on the contour of weld bead 110.

[0030] The reference line assigning unit 6 identifies a first reference line for measuring the length of the weld bead 110 and multiple second reference lines for measuring the width of the weld bead 110, starting from the end point, and assigns these to the contour data. The first reference line and the midpoint positions of each of the multiple second reference lines intersect within a predetermined angular range. As described later, multiple second reference lines are assigned along the contour of the weld bead 110 at equal first distances (equidistances) so as to intersect with the first reference line within a predetermined angular range. Here, the second reference line is assigned along the contour of at least one side of the weld bead 110 at the first distance, while two points on the contour on both sides of the weld bead 110 where the second reference line intersects are sequentially moved so that the midpoint positions of the first reference line and the multiple second reference lines intersect within a predetermined angular range. The first reference line corresponds to the weld center line. In other words, the first reference line corresponds to a reference line in the length direction of the weld bead 110. The second reference line corresponds to a reference line in the width direction of weld bead 110. Specific methods for providing the first reference line and the second reference line will be described later.

[0031] In this specification, the term "predetermined angle range" refers to 90°±δ (°), where δ is a preset fixed value, for example, 5 (°).

[0032] Dimension calculation unit 7 calculates the length of weld bead 110 based on the first reference line. Furthermore, the dimension measurement unit calculates the width of weld bead 110 at multiple locations along the first reference line based on multiple second reference lines. Note that dimension calculation unit 7 also calculates statistics related to the calculated widths of weld bead 110 at multiple locations. Examples of statistics include maximum values, minimum values, average values, and variances.

[0033] Memory unit 8 stores a welding information acquisition program that describes a method for acquiring the dimensions, positional deviation, leg length, and throat depth of weld bead 110. Memory unit 8 also stores the dimensional measurement results of weld bead 110. Memory unit 8 temporarily stores the shape data acquired by data acquisition unit 1. As will be described later, bead region extraction unit 2 performs machine learning (deep learning) using learning data to extract bead region data that indicates the formation region of the weld bead from the shape data.

[0034] The display unit 9 displays the shape data and bead region data as two-dimensional or three-dimensional image data. The display unit 9 also displays the contour data before and after the first and second reference lines are added. The display unit 9 may display the dimensional measurement results numerically or in a table format. Alternatively, the display unit 9 may display the dimensional measurement results by adding them to the contour data after the first and second reference lines are added. When displaying the dimensional measurement results for multiple weld beads 110, the display unit 9 may display the results in a graph format. The display unit 9 also performs display according to the outputs of the positional deviation calculation unit 13 and the leg length / throat depth calculation unit 16.

[0035] The learning data generating unit 10 generates learning data used to extract the bead region data, and the storage unit 8 may store the learning data in this case.

[0036] Maximum-change point identifying unit 11 identifies, for each point group having a common coordinate in the longitudinal direction of weld bead 110, a maximum-change point at which the differential coefficient of the function is maximum when the coordinate in the Z-axis direction of each measurement point is considered as a function of the coordinate in the width direction of weld bead 110 in the shape data stored in memory unit 8. Here, the width direction of weld bead 110 is one of the X-axis direction and the Y-axis direction, and the longitudinal direction of weld bead 110 is the other of the X-axis direction and the Y-axis direction. A method for identifying the width direction and the longitudinal direction of weld bead 110 will be described later.

[0037] The boundary line estimation unit 12 estimates the boundary line FL1 (see Figure 18) between the two base materials 100 based on the coordinates of the maximum change point identified by the maximum change point identification unit 11 that is outside the formation area of ​​the weld bead 110 identified by the bead area data generated by the bead area extraction unit 2.

[0038] For each of the second reference lines identified by reference line providing unit 6, misalignment amount calculation unit 13 calculates, as the misalignment amount, the distance (D1, D2) between the intersection of the second reference line with boundary line FL1 estimated by boundary line estimation unit 12 and the intersection of the second reference line with the outer edge of weld bead 110. Misalignment amount calculation unit 13 outputs the calculated misalignment amount to display unit 9.

[0039] Base metal line estimation unit 14 refers to the shape data stored in memory unit 8, and estimates base metal line FL2 (see FIGS. 27, 28, 29, and 31) that indicates the exposed surfaces (surfaces of base metal 100) of two base metals 100 in the cross section at width line WL (see FIG. 24). Width line WL is a straight line that includes one of the multiple second reference lines (reference lines in the width direction of weld bead 110 for measuring the width of weld bead 110) identified by reference line providing unit 6.

[0040] The joint determination unit 15 determines whether the intersection of the base material wires FL2 (lines indicating the surfaces of the base materials 100) of the two base materials 100 in the cross section is located at a position sandwiched between both ends of the width line WL (a line in the width direction of the weld bead 110) in the width line WL direction.

[0041] The leg length and throat depth calculation unit 16 calculates the leg length and throat depth and outputs them to the display unit 9 .

[0042] 2 is similar to the configuration of a known personal computer (PC), that is, welding information acquisition device 20 includes, as hardware, at least an input port 21, a central processing unit (CPU) 22, a graphics processing unit (GPU) 23, a random access memory (RAM) / read only memory (ROM) 24, an output port 25, and a data bus 26.

[0043] The multiple functional blocks in welding information acquisition device 20 shown in Fig. 1 correspond to the various devices shown in Fig. 2. For example, RAM / ROM 24 corresponds to storage unit 8 shown in Fig. 1. Input port 21 corresponds to data acquisition unit 1 shown in Fig. 1. In addition, welding information acquisition device 20 includes display 27 as display unit 9 shown in Fig. 1.

[0044] Welding information acquisition device 20 may include a keyboard or a touch panel (neither of which is shown) as an input device. Display 27 may be a touch panel and used as an input device. Welding information acquisition device 20 may include a hard disk drive (HDD) or a solid state drive (SSD) in addition to RAM / ROM 24 as storage unit 8 shown in FIG. 1 .

[0045] The functions of the bead region extraction unit 2, the contour extraction unit 3, the end point acquisition unit 5, the reference line assignment unit 6, the dimension calculation unit 7, the learning data generation unit 10, the maximum change point identification unit 11, the boundary line estimation unit 12, the positional deviation calculation unit 13, the base wire estimation unit 14, the joint determination unit 15, and the leg length / throat thickness calculation unit 16 shown in FIG. 1 are realized by the GPU 23 or the CPU 22 executing a welding information acquisition program stored in the storage unit 8. The function of the bead region extraction unit 2, i.e., the function of extracting the formation region of the weld bead 110 from the shape data, is realized by the GPU 23 executing a part of the welding information acquisition program. The other functions may be realized by executing a program in the GPU 23 or by executing a program in the CPU 22. The function of the learning data generation unit 10 is realized by the GPU 23 or the CPU 22 executing a program separate from the welding information acquisition program.

[0046] The GPU 23 and the CPU 22 may be collectively referred to as a processor. The welding information acquisition device 20 may have one processor (GPU 23 or CPU 22). However, from the viewpoint of improving the data processing speed, it is preferable that the welding information acquisition device 20 have multiple processors (GPU 23 and CPU 22, or there may be multiple of each).

[0047] Although FIG. 2 shows an example in which various devices are connected to one data bus 26, multiple data buses 26 may be provided depending on the purpose, as in a normal PC.

[0048] [2: Procedure for Measuring the Dimensions of a Weld Bead] Fig. 4 is a flowchart showing the procedure for measuring the dimensions of a weld bead. Fig. 5 is a schematic diagram showing the outline of a weld bead during smoothing processing and when edge points are acquired.

[0049] First, the shape data acquired by the appearance inspection device 30 is input to the data acquisition unit 1 of the welding information acquisition device 20 as shown in FIG. 4 (step S1).

[0050] Next, the bead region extraction unit 2 extracts bead region data from the shape data (step S2). Step S2 includes a number of steps, the details of which will be explained later.

[0051] Next, the contour extraction unit 3 extracts contour data from the bead region data (step S3). Specifically, the contour data is extracted from the bead region data using a library having an image processing function, such as OpenCV (Open Source Computer Vision Library).

[0052] Furthermore, the smoothing processor 4 performs a smoothing process on the contour data (step S4). Specifically, as shown in Fig. 5, a moving average filter is applied to the contour data to remove jaggies contained in the contour data. Note that jaggies are a type of noise that occurs in digital images, and refer to step-like jaggedness that appears on lines and contours.

[0053] The smoothing process may be performed by a method other than the filtering process using a moving average filter.

[0054] Next, the endpoint acquisition unit 5 acquires one or more endpoints (see FIG. 5) from the contour data after step S4 has been executed (step S5). The endpoints are selected, for example, as points at which the contour inflects, in other words, points at which the contour slope changes in the opposite direction (points at which the sign of the change in Y coordinate relative to the change in X coordinate, or points at which the sign of the change in X coordinate relative to the change in Y coordinate). Note that if smoothing processing is not performed, the corners of jaggies may be mistakenly recognized as endpoints. By performing the smoothing processing in step S4, it is possible to prevent erroneous recognition of endpoints.

[0055] Furthermore, the reference line providing unit 6 provides the aforementioned first and second reference lines to the contour data after the endpoint acquisition in step S5 has been performed, starting from the endpoints (step S6). In other words, the reference line providing unit 6 provides the contour data with a first reference line for measuring the length of weld bead 110 and multiple second reference lines for measuring the width of weld bead 110, starting from the endpoints. The first reference line and the multiple second reference lines are specified so that each of the multiple second reference lines intersects with the first reference line at an intersection of the first reference line and the second reference line within a predetermined angle range. Note that step S6 includes multiple steps. Details of steps S5 and S6 will be described later.

[0056] Next, dimension calculation unit 7 calculates the length of weld bead 110 based on the first reference line. Dimension calculation unit 7 also calculates the width of weld bead 110 at multiple locations along the first reference line based on multiple second reference lines. Dimension calculation unit 7 then calculates the aforementioned statistics for the widths of weld bead 110 at multiple locations whose dimensions have been calculated (step S7).

[0057] If multiple endpoints are acquired in the endpoint acquisition in step S5, multiple pairs of a first reference line and multiple second reference lines are generated in step S6. In this case, in the dimension calculation in step S7, the length and width of weld bead 110 are calculated for each pair, and the value of the pair that results in the longest length of weld bead 110 is selected as the final calculation result. Specifically, the maximum value of the multiple calculated lengths of weld bead 110 is set as the final length of weld bead 110, and only the first reference line corresponding to this is selected. Also, only the multiple second reference lines corresponding to the selected first reference line are selected. Based on the selected multiple second reference lines, the width of weld bead 110 is calculated at multiple locations along the selected first reference line. The remaining calculation results, i.e., calculation results based on first reference lines that do not have a final length and the corresponding second reference lines, are not stored in memory unit 8 but are deleted.

[0058] The dimensions of weld bead 110 calculated in the dimension calculation in step S7 are output (output of calculation results in step S8). The calculation results are output to memory unit 8 or to a device or memory unit (neither of which are shown) provided outside welding information acquisition device 20. In the latter case, the calculation results are transmitted via output port 25. Alternatively, the calculation results may be output directly to display unit 9. In this case, the calculation results are displayed on display unit 9 in the form of numerical values, a table, or a graph. The calculation results may also be displayed on display unit 9 together with contour data to which the first and second reference lines have been added.

[0059] 2-1: Procedure for Extracting Weld Bead Formation Region FIG. 6 is a flowchart showing the procedure for extracting the weld bead formation region, which corresponds to the bead region data extraction process in step S2 of FIG.

[0060] Fig. 7 is a diagram showing an image of shape data and a weld bead formation region and other regions in the shape data. Fig. 8 is a flowchart showing an annotation procedure for generating learning data. Fig. 9A is an example showing how annotations are added to shape data, and Fig. 9B is an example showing an image of the shape data after annotations are added.

[0061] 6 , when extracting the formation region of weld bead 110 from the shape data as bead region data, shape data to be extracted is prepared. Furthermore, when performing machine learning (deep learning) in step S12, the required number and types of learning data are prepared (step S11). As described above, the learning data may be data stored in storage unit 8, or data input from outside welding information acquisition device 20.

[0062] Next, the bead region extraction unit 2 uses the learning data to perform machine learning (deep learning) on ​​an algorithm for extracting bead region data (step S12). Using the reinforced learning algorithm, the shape data is processed into a state where it is divided into bead region data and data related to other regions (such as the base material 100 region) (step S13). An algorithm called semantic segmentation is used to perform steps S12 and S13. Semantic segmentation is an algorithm that associates labels or categories with data for all pixels in image data and is used for object detection, etc. In this embodiment, an algorithm called DeepLab v3+ is specifically used, but is not limited to this. The algorithm is executed by the GPU 23.

[0063] When the shape data is viewed as image data acquired by an imaging element, classification is performed on the data of each pixel in the shape data. In this embodiment, the data of each pixel in the shape data is combined with the aforementioned color information and distance information representing the distance from the three-dimensional shape measurement sensor 31. In steps S12 and S13, classification is performed based on this color information. Furthermore, the shape data is expanded, reduced, or divided to a fixed size and then input into a convolutional neural network (CNN) implemented in the GPU 23, specifically DeepLab v3+. Furthermore, arithmetic processing is performed on the shape data in each layer of the CNN. At this time, classification is performed based on the color information or distance information, dividing the data into two classes: one corresponding to the formation region of the weld bead 110 and the other corresponding to the other regions. The bead region extraction unit 2 generates shape data to which the aforementioned classes are assigned, in a format in which the bead region data and the other regions can be distinguished from each other. Specifically, as shown in FIG. 7, an image is generated in which the area where weld bead 110 is formed and the other areas are color-coded for the image of the shape data.

[0064] After step S13 is executed, the processing result is stored in the storage unit 8 (storing the processing result in step S14).

[0065] In this way, by using an algorithm that has been reinforced by machine learning (deep learning), it is possible to accurately and reliably extract bead region data, which is the formation region of weld bead 110, from the shape data. Furthermore, because deep learning is used to identify the bead region data, problems such as unstable extraction accuracy of bead region data due to delicate parameter settings do not occur, compared to when, for example, feature amounts of the external shape of a weld are identified by noise removal using a low-pass filter or the like.

[0066] The learning data is generated based on shape data obtained by visually inspecting a separately prepared actual weld bead 110. Here, an example will be described in which the learning data is generated by learning data generation unit 10 provided inside welding information acquisition device 20. However, as described above, the learning data may also be generated outside welding information acquisition device 20.

[0067] First, as shown in Fig. 8, a figure designation mode for adding an annotation is selected (step S21). In this embodiment, the shape of weld bead 110 is approximated by a polygon and an annotation is added.

[0068] Next, the training data generation unit 10 calls up the shape data for generating the training data, opens the data file of the shape data (step S22), and further displays and visualizes the opened data file on the display unit 9. In this case, the display unit 9 is a touch panel and also has an input function. Note that the input function may be realized by another device such as a touch pen or a mouse.

[0069] After the image of the shape data is displayed on display unit 9, the worker traces the outline of weld bead 110 while viewing the image of the shape data. In this embodiment, the polygon designation mode described above is selected as the shape designation mode. In this case, as shown in FIG. 9A , the vertices of the polygon are determined by sequentially clicking along the outline with a touch pen or the like. Finally, as shown in FIG. 9B , weld bead 110 is surrounded by a polygon (the vertices of the polygon), the formation area of ​​weld bead 110 is designated, and the annotation is completed (step S23).

[0070] The shape data to which annotations have been added is stored as learning data in the storage unit 8 (step S24).

[0071] Next, the learning data generation unit 10 determines whether or not any data files of shape data to be annotated remain (step S25). If the determination result of step S25 is negative, that is, if no data files of shape data to be annotated remain, the annotation work is terminated. If the determination result of step S25 is positive, that is, if any data files of shape data to be annotated remain, the series of processes of steps S22 to S24 are repeatedly executed until the determination result of step S25 becomes negative. Note that the process of step S25 may be executed manually.

[0072] [2-2: Procedure for assigning the first and second reference lines] Figure 10 is a flowchart showing the procedure for assigning the first and second reference lines, which corresponds to the processing of step S6 shown in Figure 4 (generating the first and second reference lines and assigning them to the contour data).

[0073] Fig. 11A is a schematic diagram showing a first loop process when providing a first reference line and a second reference line. Fig. 11B is a schematic diagram showing a process subsequent to the process shown in Fig. 11A. Fig. 11C is a schematic diagram showing a process subsequent to the process shown in Fig. 11B. Fig. 12 is a schematic diagram showing a second loop process when providing a first reference line and a second reference line.

[0074] First, the endpoint acquired by the endpoint acquisition unit 5 is set to endpoint A on the contour data. ０As shown in FIGS. 10 and 11A, the end point A ０ Imaginary point B on top ｉ , C ｉ (Step S31). Although i is an integer of 0 or 1 or more, in Step S31, i=0. ｉ , C ｉ are the endpoints A and B, respectively. ０ The process of step S31 is sometimes called an initial setting process. The processes from step S31 onward are executed by the reference line providing unit 6.

[0075] Next, as shown in FIG. 11B, the virtual point B ｉ along the contour of the weld bead 110, ０ The point after the movement is moved by a predetermined distance (hereinafter referred to as the first distance) in the first direction, which is the direction away from the virtual point B. （ｉ＋１） At the same time, the virtual point C ｉ along the contour of the weld bead 110, ０ The point after the movement is referred to as imaginary point C. （ｉ＋１） (Step S32). （ｉ＋１） and virtual point C （ｉ＋１） The midpoint of Ａ（ｉ＋１） (Step S33). The processes of Steps S32 and S33 may be referred to as a first loop process. As shown in FIG. 11B and FIG. 11C following FIG. 11B, the first loop process is sequentially performed to set the endpoint A ０ and virtual point B １ , B ２ , C １ , C ２ is located on the contour of the weld bead 110 at the midpoint A １ , A ２ are respectively placed within the region of the weld bead 110. ０ -A １ -A ２ forms part of the first reference line. １ -C １ and line segment B ２ -C ２correspond to the second reference line.

[0076] Next, the line segment B on the first reference line （ｉ＋１） -C （ｉ＋１） and line segment A on the second reference line ｉ -A （ｉ＋１） It is determined whether or not the line segment B intersects with the line segment B within the predetermined angle range (step S34). （ｉ＋１） -C （ｉ＋１） and line segment A ｉ -A （ｉ＋１） It is determined whether or not the angles intersect within a range of 90°±δ(°).

[0077] If the result of the determination in step S34 is negative, that is, if the line segment B （ｉ＋１） -C （ｉ＋１） and line segment A ｉ -A （ｉ＋１） If the angles intersect beyond the predetermined angle range, the process proceeds to step S36.

[0078] In step S36, the virtual point B （ｉ＋１） and virtual point C （ｉ＋１） along the contour of the weld bead 110 and at the end point A ０ The point after the movement is moved by a first distance in the direction away from the virtual point B. （ｉ＋２） or virtual point C （ｉ＋２） and proceeds to step S37.

[0079] In step S37, the line segment B （ｉ＋２） -C （ｉ＋１） and line segment A ｉ -A （ｉ＋１） The intersection angle with line segment B （ｉ＋１） -C （ｉ＋２） and line segment A ｉ -A （ｉ＋１） The intersection angle between the virtual point B and the virtual point B is compared, and the intersection angle is within or close to a predetermined angle range. （ｉ＋２） or virtual point C （ｉ＋２） Select .

[0080] The selected virtual point is virtual point B （ｉ＋２） In this case, the midpoint A （ｉ＋１） Virtual point B （ｉ＋２） and virtual point C （ｉ＋１） The original virtual point B is corrected to the midpoint of （ｉ＋１）Skip and set virtual point B （ｉ＋２） New virtual point B （ｉ＋１） Set it as follows.

[0081] On the other hand, the selected virtual point is the moving virtual point C （ｉ＋２） In this case, the midpoint A （ｉ＋１） Move to point B （ｉ＋１） and moving point C （ｉ＋２） The original moving point C is corrected to the midpoint of （ｉ＋１） Skip and set the virtual point C （ｉ＋２） New virtual point C （ｉ＋１） After step S37 is completed, the process returns to step S34, and the series of processes in steps S36 and S37 are repeatedly executed until the determination result in step S34 becomes affirmative. The processes in steps S36 and S37 are sometimes collectively referred to as the second loop process. As shown in FIG. 12, by sequentially proceeding with the second loop process, the endpoint A ０ and virtual point B １ , B ２ , C １ , C ２ is located on the contour of the weld bead 110 at the midpoint A １ , A ２ are respectively placed within the area of ​​the weld bead 110. ０ -A １ -A ２ - ... -A ｉ forms part of the first reference line. １ -C １ , B ２ -C ２ , ..., B ｉ -C ｉ 12, for example, the virtual point B 1 is skipped when the determination result of step S34 is negative. ｉ The traces are indicated by dotted circles.

[0082] On the other hand, if the result of the determination in step S34 is affirmative, that is, if the line segment B （ｉ＋１） -C （ｉ＋１） and line segment A ｉ -A （ｉ＋１） If the two intersect within a predetermined angle range, then path A ０ -B ｉ Length of and route A０ -C ｉ It is determined whether or not the sum of the lengths of the two exceeds the length of the contour of weld bead 110 (step S35).

[0083] If the determination result in step S35 is negative, that is, route A ０ -B ｉ Length of and route A ０ -C ｉ If the sum of the lengths of the two does not exceed the length of the contour of weld bead 110, variable i is counted up to (i+1), and the process returns to step S32, and the series of processes from step S32 to S34 are repeatedly executed until the determination result of step S35 becomes positive. ０ -B ｉ The length of ０ From virtual point B ｋ (k is an integer, 1≦k<i) via virtual point B ｉ Route A refers to the distance from the destination to the destination. ０ -C ｉ The length of ０ From virtual point C ｋ via virtual point C ｉ refers to the distance of the route to

[0084] On the other hand, if the determination result in step S35 is affirmative, that is, route A ０ -B ｉ Length of and route A ０ -C ｉ If the sum of the lengths of the end points A and B exceeds the length of the contour of the weld bead 110, the first and second reference lines are provisionally determined (step S38). ０ The starting point is a plurality of midpoints A set at the end of step S35. １ , ..., Ai. At the end of step S35, the end point of the first reference line reaches the contour of weld bead 110 or intersects with the contour. １ -C １ , ..., B ｉ -C ｉ The number of second reference lines is determined by the ratio of the length of the weld bead 110 to the first distance (the distance from the end point A０ The distance is determined according to the distance (a predetermined distance in the first a direction, which is a direction away from the target point).

[0085] However, in the process up to step S38, the first reference line may not be set appropriately. Therefore, in this embodiment, a correction process for the first reference line (step S39) described below is uniformly performed after execution of step S38 to determine the first reference line and the plurality of second reference lines. The process of step S39 will be further described.

[0086] 2-3: Correction Procedure for the First Reference Line FIG. 13 is a flowchart showing the correction procedure for the first reference line, which corresponds to the process of step S39 shown in FIG.

[0087] FIG. 14A is an example of contour data to which a first reference line has been added when no correction is performed, and FIG. 14B is another example of contour data to which a first reference line has been added when no correction is performed.

[0088] FIG. 15A is an example of contour data when the first reference line is transformed into a tentative reference line, and FIG. 15B is another example of contour data when the first reference line is transformed into a tentative reference line.

[0089] FIG. 16A is an example of contour data to which a first reference line has been added after correction has been completed, and FIG. 16B is another example of contour data to which a first reference line has been added after correction has been completed.

[0090] 14A, 15A, and 16A show the same contour data, and similarly, 14B, 15B, and 16B show the same contour data.

[0091] When bead region data is extracted from shape data by the bead region data extraction process in step S2 shown in Fig. 4, specifically, by the procedure shown in Fig. 6 (step S13), noise may be included in the extracted bead region data (see Fig. 14A). In such a case, when the length of weld bead 110 is calculated according to the procedure after contour data acquisition in step S3 shown in Fig. 4, a value different from the actual length may be output. Alternatively, if the endpoints are not properly acquired in endpoint acquisition in step S5 shown in Fig. 4, the first reference line itself may not be properly generated (see Fig. 14B).

[0092] Even in these cases, the first reference line can be properly added to the contour data by performing the procedure described below. This also allows a plurality of second reference lines to be properly added to the contour data. Furthermore, the length and width of weld bead 110 can be accurately calculated. The actual correction procedure will be described in detail below.

[0093] First, a predetermined length is removed from each end of the first reference line to set a tentative reference line (step S41 in FIG. 13; see also FIGS. 15A and 15B). In this case, it is not determined whether the predetermined length has been correctly removed from each end of the first reference line.

[0094] Next, straight lines (hereinafter sometimes referred to as extension lines) according to the respective slopes near both ends of the provisional reference line are extended from each end of the provisional reference line until they intersect with the contour (step S42 in FIG. 13; also see FIGS. 16A and 16B).

[0095] Furthermore, a new first reference line is created by combining the provisional reference line with extension lines extending from both ends of the provisional reference line, and the new first reference line is added to the outline data (step S43 in FIG. 13; see also FIGS. 16A and 16B).

[0096] In the dimension calculation in step S7 shown in FIG. 4, the length of the first reference line shown in FIG. 16A or FIG. 16B is calculated as the length of weld bead 110.

[0097] It should be noted that, when multiple endpoints are acquired in acquiring endpoints in step S5 shown in Fig. 4, first and second reference lines are assigned to the contour data using each endpoint as a starting point in the procedure shown in Fig. 10, and the first reference line is corrected in the procedure shown in Fig. 13. As described above, the length and width of weld bead 110 are calculated based on the first reference line having a fixed length and the multiple second reference lines corresponding to it.

[0098] Hereinafter, the corrected first reference line is used as the weld center line to measure the positional deviation, leg length, and throat depth. Although there is a possibility that it may be adversely affected by noise, etc., the first reference line before correction may be used as the weld center line to measure the positional deviation, leg length, and throat depth without correction. In any case, the weld center line used to measure the positional deviation, leg length, and throat depth includes at least a portion of the first reference line before correction.

[0099] 3: Procedure for Measuring Positional Deviation of Weld Bead FIG. 17 is a flowchart showing the procedure for measuring positional deviation of weld bead 110 formed linearly.

[0100] First, maximum change point identifying unit 11 identifies the longitudinal direction and width direction of weld bead 110 (step S51). The detailed identification procedure will be described later.

[0101] Next, maximum-change point identifying unit 11 regards the Z coordinate of each measurement point as a function of the width-direction coordinate of weld bead 110 and calculates the differential coefficient (differential value) of this function for each measurement point (step S52). For example, if the width direction is the Y-axis direction, the differential coefficient f'(y) of the function z = f(y) is calculated for each measurement point, where z is the Z coordinate of each measurement point and y is the Y coordinate. FIG. 18 is an explanatory diagram illustrating a procedure for estimating boundary line FL1 between two base materials 100. In FIG. 18, the horizontal direction (lengthwise position) corresponds to the longitudinal direction of weld bead 110 identified in step S51, and the vertical direction (widthwise position) corresponds to the width direction of weld bead 110 identified in step S51. In FIG. 18, the boundary between the formation region of weld bead 110 and other regions identified by the bead region data generated by bead region extracting unit 2 is indicated by a dashed line. In FIG. 19, the height of the surface of the base material 100 along line AA in FIG. 18 is indicated by a broken line, and the differential coefficient (differential value) along line AA is indicated by a solid line.

[0102] Then, maximum change point identifying unit 11 identifies the maximum change point at which the differential coefficient of the function is maximum for each point group having the same coordinate in the longitudinal direction of weld bead 110 (step S53). In Figure 18, the identified maximum change points are indicated by black dots.

[0103] Next, boundary line estimation unit 12 estimates a boundary line (fitting line) FL1 between two base materials 100 based on the coordinates of a plurality of maximum change points that are outside the formation region of weld bead 110 and are identified by the bead region data generated by bead region extraction unit 2, among the maximum change points identified in step S53 (step S54). In Fig. 18, the maximum change points in the range indicated by R1 are maximum change points that are inside the formation region of weld bead 110, and the maximum change points in the range indicated by R2 are maximum change points that are outside the formation region of weld bead 110. Therefore, boundary line estimation unit 12 estimates boundary line FL1 between two base materials 100 based on the coordinates of a plurality of maximum change points that are in the range indicated by R2.

[0104] Boundary line estimation unit 12 estimates boundary line FL1 of base material 100 so that the sum of squares of the distances between a plurality of maximum change points outside the formation region of weld bead 110 and boundary line FL1 of base material 100 is minimized. In other words, boundary line estimation unit 12 estimates boundary line FL1 of two base materials 100 by the least squares method.

[0105] Then, the positional deviation amount calculation unit 13 identifies the intersections of the second reference lines identified by the reference line providing unit 6 with the boundary line FL1 estimated in step S54 (step S55).

[0106] Next, for each of the plurality of second reference lines identified by reference line providing unit 6, misalignment amount calculation unit 13 calculates distances D1 and D2 between the intersection (boundary line FL1) identified in step S55 and the intersection with the outer edge of the formation area of ​​weld bead 110 identified by the bead area data, as the misalignment amount, and outputs the calculated distances to display unit 9 (step S56). Note that misalignment amount calculation unit 13 may be configured to determine that there is no abnormality when the calculated misalignment amount is within a predetermined numerical range, and to determine that there is an abnormality when the calculated misalignment amount is not within the predetermined numerical range.

[0107] In this way, the boundary line FL1 between two base materials 100 is estimated based on the coordinates of multiple maximum change points outside the formation region of weld bead 110, which makes it possible to prevent the estimated boundary line FL1 from deviating significantly from the actual boundary line due to melting of base material 100 or the generation of spatter at the outer edge of weld bead 110. Furthermore, the boundary line FL1 between two base materials 100 is estimated by the least squares method based on the coordinates of three or more maximum change points outside the formation region of weld bead 110, which makes it possible to further reduce the deviation between the estimated boundary line FL1 and the actual boundary line.

[0108] 20 is a flowchart showing the procedure for specifying the longitudinal and width directions of weld bead 110. That is, Fig. 20 explains the processing executed in step S51.

[0109] In order to identify the longitudinal and width directions of the weld bead 110, the maximum change point identification unit 11 first acquires the coordinates of all measurement points that constitute the first reference line (weld center line) identified in step S6 (step S511).

[0110] Next, the maximum change point identification unit 11 converts the coordinates acquired in step S511 so that the ratio between the difference in the X-coordinate values ​​of adjacent measurement points in the X-axis direction and the difference in the Y-coordinate values ​​of adjacent measurement points in the Y-axis direction is the same as the ratio between the actual spacing between adjacent measurement points in the X-axis direction and the actual spacing between adjacent measurement points in the Y-axis direction (step S512). Specifically, as shown on the left side of FIG. 21 , in the coordinates before conversion, the difference between the X-coordinate values ​​of adjacent measurement points in the X-axis direction and the difference between the Y-coordinate values ​​of adjacent measurement points in the Y-axis direction are both 1 (the difference between the grid coordinate axes is 1). However, the actual spacing between adjacent measurement points in the X-axis direction is 0.025 mm, and the actual spacing between adjacent measurement points in the Y-axis direction is 0.1 mm. In other words, the ratio between the difference in the X-coordinate values ​​of adjacent measurement points in the X-axis direction and the actual spacing between the Y-coordinate values ​​of adjacent measurement points in the Y-axis direction is 1:4. Therefore, as shown in Fig. 21 , the coordinates are converted so that the ratio of the difference in the X-coordinate values ​​of adjacent measurement points in the X-axis direction to the difference in the Y-coordinate values ​​of adjacent measurement points in the Y-axis direction is 1:4. Specifically, the Y-coordinate value of each measurement point is multiplied by 4. For example, as shown in Fig. 21 , if the X- and Y-coordinates of measurement points P1, P2, and P3 constituting the first reference line are (0,0), (1,1), and (2,2) before conversion, the X- and Y-coordinates of measurement points P1, P2, and P3 constituting the first reference line after conversion become (0,0), (1,4), and (2,8).

[0111] Next, the maximum-change point specifying unit 11 specifies a circumscribing rectangle SQ of the measurement points P1, P2, and P3 that form the first reference line, as shown in FIG. 21 (step S513).

[0112] Then, maximum-change point identification unit 11 determines whether the horizontal length (X-axis direction) of circumscribing rectangle SQ is longer than the vertical length (Y-axis direction) of circumscribing rectangle SQ (step S514). If the horizontal length (X-axis direction) of circumscribing rectangle SQ is longer than the vertical length (Y-axis direction), maximum-change point identification unit 11 identifies the horizontal direction (X-axis direction) as the longitudinal direction of weld bead 110 and the vertical direction (Y-axis direction) as the width direction of weld bead 110 (step S515). On the other hand, if the vertical length (Y-axis direction) of circumscribing rectangle SQ is longer than the horizontal length (X-axis direction), maximum-change point identification unit 11 identifies the horizontal direction (X-axis direction) as the width direction (first direction) of weld bead 110 and the vertical direction (Y-axis direction) as the longitudinal direction (second direction) of weld bead 110 (step S516). The width direction (first direction) of the weld bead 110 is perpendicular to the direction in which the measurement sensor 31 faces, and the longitudinal direction (second direction) of the weld bead 110 is perpendicular to the direction in which the measurement sensor 31 faces and the width direction (first direction) of the weld bead 110.

[0113] In this way, in step S51, maximum change point identification unit 11 identifies the direction in which measurement points P1, P2, and P3 that make up the first reference line are distributed more narrowly, of the X-axis direction and the Y-axis direction, as the width direction of weld bead 110, and identifies the direction in which measurement points P1, P2, and P3 that make up the first reference line are distributed more widely, as the longitudinal direction of weld bead 110.

[0114] 22 illustrates an example of the distribution of measurement points that form the first reference line. In FIG. 22, the horizontal axis represents the X coordinates of the measurement points after conversion in step S512, and the vertical axis represents the Y coordinates of the measurement points after conversion in step S512. In this example, the measurement points are distributed more widely in the Y axis direction (the scanning direction of three-dimensional shape measurement sensor 31) than in the X axis direction (the direction perpendicular to the scanning direction of three-dimensional shape measurement sensor 31). Therefore, maximum-change point identification unit 11 identifies the Y axis direction as the longitudinal direction of weld bead 110 and the X axis direction as the width direction of weld bead 110.

[0115] In this manner, in this embodiment, the longitudinal direction and width direction of weld bead 110 can be automatically identified.

[0116] [4: Procedure for Measuring Leg Length and Throat Depth of Weld Bead] FIG. 23 is a flowchart showing the procedure for measuring the leg length and throat depth of weld bead 110.

[0117] First, in the XY plane, base wire estimation unit 14 identifies, as a width line WL, a straight line including any of the second reference lines identified by reference line providing unit 6, as shown in Fig. 24 (step S61). Note that calculation unit 28 executes the processes of step S61 and steps S62 to S68 described below for all of the second reference lines identified by reference line providing unit 6. In Fig. 24, symbol BR indicates the formation region of weld bead 110 identified by the bead region data.

[0118] Next, the base wire estimation unit 14 refers to the shape data stored in the storage unit 8 and acquires the Z coordinates of the constituent points constituting the width line WL identified in step S61 (step S62). In FIG. 25, constituent points that are measurement points are indicated by black circles, and constituent points that are not measurement points are indicated by square dots. The constituent points are arranged at intervals of 0.025 mm in the X-axis direction. In other words, the difference in X-coordinate values ​​between adjacent constituent points is 1 (the difference between squares on the grid coordinate axis is 1). The constituent points also include the measurement point. The base wire estimation unit 14 acquires the Z coordinates of constituent points that are measurement points from the shape data stored in the storage unit 8. The base wire estimation unit 14 sets the Z coordinate of a constituent point that is not a measurement point to the Z coordinate of a measurement point (indicated by a hollow circle in FIG. 25) that has the same X coordinate as the constituent point and is closest to the constituent point in the Y-axis direction. Fig. 26 illustrates the coordinates of the constituent points acquired in step S62 (a table representing the graph in Fig. 25). In the example of Fig. 26, the base wire estimation unit 14 acquires the Z coordinates of the constituent points whose XY coordinates are (0,4), (1,3.667), (2,3.333), (3,3), (4,2.667), (5,2.333), (6,2), (7,1.667), (8,1.333), and (9,1). As the Z coordinates of the constituent points whose XY coordinates are (0,4), (3,3), (6,2), and (9,1), the Z coordinates of the measurement points whose XY coordinates are (0,4), (3,3), (6,2), and (9,1) can be acquired from the shape data. That is, the Z coordinate of a constituent point whose XY coordinate is (1, 3.667) is set to the Z coordinate of the measurement point whose X coordinate is equal to that of the constituent point and is closest to that constituent point in the Y-axis direction (shown by a hollow circle in FIG. 25 ), i.e., the Z coordinate of the measurement point whose XY coordinate is (1, 4). Similarly, the Z coordinates of constituent points whose XY coordinates are (2, 3.333), (4, 2.667), (5, 2.333), (7, 1.667), and (8, 1.333) are set to the Z coordinates of measurement points whose XY coordinates are (2, 3), (4, 3), (5, 2), (7, 2), and (8, 1). In this way, the base wire estimation unit 14 sets the Z coordinate of a constituent point that is not a measurement point to the Z coordinate of the measurement point whose X coordinate is equal to that of the constituent point and is closest to that constituent point in the Y-axis direction. Therefore, even if there are few measurement points included in the width line WL, a more detailed cross-sectional shape of the width line WL can be obtained.

[0119] Next, the base wire estimation unit 14 converts the X, Y, and Z coordinates of the constituent points acquired in step S62 into X, Y, and Z coordinates that indicate the actual positional relationship of the constituent points (step S63). The measurement points are arranged at regular intervals of 0.025 mm in the X-axis direction and at regular intervals of 0.1 mm in the Y-axis direction. Therefore, for example, by multiplying the X coordinate by 0.025 and the Y coordinate by 0.1, the X, Y, and Z coordinates that indicate the actual positional relationship of the constituent points can be acquired.

[0120] Next, as shown in Fig. 27 , the base metal wire estimation unit 14 acquires a base metal wire (fitting straight line) FL2 indicating the exposed surfaces (surfaces of the base metal 100) of the two base metals 100 in the cross section along the width line WL identified in step S61 based on the X- and Z-coordinates of constituent points outside the formation region of the weld bead 110 identified by the bead region data, among the X-, Y-, and Z-coordinates acquired in step S63 (step S64). That is, in the example of Fig. 27 , the base metal wire FL2 is acquired based on the X- and Z-coordinates of constituent points in the region indicated by RF (the region outside the formation region of the weld bead 110 on the width line WL). In Figs. 27 , 28 , 29 , and 31 , the symbol ML indicates the exposed surfaces of the base metal 100 and the weld bead 110. In Fig. 27 , the symbol BB indicates a point corresponding to the boundary (outer edge) of the weld bead 110 identified by the bead region data. Here, the base metal wire estimation unit 14 acquires the base metal wire FL2 so that the sum of the squares of the distances in the Z-axis direction between the base metal wire FL2 and a constituent point on the width line WL outside the formation region (region RF) of the weld bead 110 is minimized. In other words, the base metal wire estimation unit 14 acquires the base metal wire FL2 by the least squares method.

[0121] In this way, the base metal wire estimation unit 14 acquires the base metal wire FL2 indicating the exposed surface (base metal surface) of the base metal 100 by referring to the Z coordinate of a measurement point that is outside the formation area of ​​the weld bead 110 identified by the bead area data, has the same X coordinate as a constituent point of the width line WL that is not a measurement point, and is closest to the constituent point in the Y-axis direction.

[0122] 28 , the base metal wire estimation unit 14 translates the base metal wire FL2 estimated in step S65 so that the base metal wire FL2 passes through a boundary (outer edge) BB of the formation region of the weld bead 110 specified by the bead region data (step S65). The base metal wire FL2 after the translation becomes the base metal wire estimated by the base metal wire estimation unit 14.

[0123] Thereafter, the joint determination unit 15 determines whether or not the intersection IS of the base wire FL2 of the two base materials 100 is located between both ends of the width line WL in the cross section (X-Z cross section) of the width line WL (step S66). In other words, the joint determination unit 15 determines whether or not both ends of the width line WL are located on either side of the intersection IS in the width line WL direction. In FIG. 27 , the horizontal direction (X-axis direction) corresponds to the width line direction. In FIG. 27 , the intersection IS is located between both ends of the width line WL from both sides in the horizontal direction, so it can be said that the intersection IS is located between both ends of the width line WL in the width line WL direction. As shown in FIG. 27 , when two base materials 100 form a T-joint, the intersection IS of the base wire FL2 is located between both ends of the width line WL in the width line WL direction. 29, when two base materials 100 form a lap joint, the intersection of the base material wire FL2 does not exist in a position sandwiched between both ends of the width line WL in the width line WL direction. Therefore, the joint determination unit 15 can determine whether the two base materials 100 form a T-joint or a lap joint through this determination.

[0124] Next, if it is determined in step S66 that the two base materials 100 constitute a T-joint, the leg length / throat depth calculation unit 16 calculates the first and second leg lengths L1 and L2 and the throat depth TH1 shown in FIG. 27 (step S67). The first leg length L1 is the distance between one point BB corresponding to the boundary (outer edge) of the weld bead 110 and the intersection point IS of the base material wire FL2 in the cross section taken along the width line WL. The second leg length L2 is the distance between the other point BB corresponding to the boundary (outer edge) of the weld bead 110 and the intersection point IS of the base material wire FL2 in the cross section taken along the width line WL. Here, a first straight line CL11 is defined as a line connecting points BB corresponding to the boundary (outer edge) of the weld bead 110 identified by the bead region data in the cross section taken along the width line WL. The second straight line CL12 is defined as a line obtained by translating the first straight line CL11 so as to pass through the lowest point in the cross section of the width line WL (the point farthest from the three-dimensional shape measuring sensor 31 in the Z-axis direction). The throat thickness TH1 is the shorter of the distance between the first straight line CL11 and the intersection point IS of the base wire FL2 and the distance between the second straight line CL12 and the intersection point IS in the cross section of the width line WL.

[0125] On the other hand, if it is determined in step S66 that the two base materials 100 constitute a lap joint, the leg length / throat thickness calculation unit 16 calculates the first and second leg lengths L3, L4 and the throat thickness TH2 shown in Fig. 29 (step S67). Here, a perpendicular line PL is drawn from the point HP with the largest Z coordinate (the point with the shortest distance from the three-dimensional shape measurement sensor 31, the point with the highest bead height) on the cross section of the width line WL to the lower base material wire FL2 (the base material wire FL2 farther from the three-dimensional shape measurement sensor 31), and the intersection of the perpendicular line PL and the higher base material wire FL2 (the base material wire FL2 closer to the three-dimensional shape measurement sensor 31) is defined as a first intersection point CP1. Also, the intersection of the perpendicular line PL and the higher base material wire FL2 (the base material wire FL2 closer to the three-dimensional shape measurement sensor 31) is defined as a second intersection point CP2. Here, the third straight line CL21 is defined as a line connecting point BB, which corresponds to the boundary (outer edge) of the weld bead 110 on the lower base wire FL2 side in the cross section of the width line WL, with the second intersection point CP2. The fourth straight line CL22 is defined as a line obtained by translating the third straight line CL21 so as to pass through the lowest point in the cross section of the width line WL (the point farthest from the three-dimensional shape measurement sensor 31 in the Z-axis direction). The throat thickness TH2 is the shorter of the distance between the third straight line CL21 and the first intersection point CP1 and the distance between the fourth straight line CL22 and the first intersection point CP1 in the cross section of the width line WL. The first leg length L3 is the distance between the first and second intersection points CP1 and CP2. The second leg length L4 is the distance between the first intersection point CP1 and a point BB corresponding to the boundary (outer edge) of the weld bead 110 on the lower base metal wire FL2 side.

[0126] Next, the leg length / throat thickness calculation unit 16 outputs the calculated throat thicknesses TH1, TH2 and the first and second leg lengths L1 to L4 to the display unit 9. The leg length / throat thickness calculation unit 16 may determine the presence or absence of an abnormality by comparing the calculated throat thicknesses TH1, TH2 and the first and second leg lengths L1 to L4 with corresponding predetermined thresholds. Specifically, the leg length / throat thickness calculation unit 16 may determine that an abnormality exists when any one of the calculated throat thicknesses TH1, TH2 and the first and second leg lengths L1 to L4 exceeds the corresponding threshold, and may determine that no abnormality exists when none of the calculated values ​​exceeds the corresponding threshold.

[0127] Figure 30 illustrates examples of throat thicknesses TH1, TH2 and first and second leg lengths L1 to L4 obtained visually and by the welding information acquisition device 20 of this embodiment, and the differences between these values, in each case where two base materials 100 form a lap joint and a T-joint.

[0128] In the example of Figure 30, when two base materials 100 form a lap joint, a first leg length L3 of 2.27 mm, a second leg length L4 of 5.84 mm, and a throat thickness TH2 of 2.10 mm are obtained by visual inspection. Furthermore, the welding information acquisition device 20 according to this embodiment obtains a first leg length L3 of 2.28 mm, a second leg length L4 of 5.41 mm, and a throat thickness TH2 of 2.02 mm. Therefore, the difference between the first leg length L3 obtained by visual inspection and the welding information acquisition device 20 is 0.01 mm, the difference between the second leg length L4 is 0.45 mm, and the difference between the throat thickness TH2 is 0.08 mm. On the other hand, when two base materials 100 form a T-joint, a first leg length L1 of 5.19 mm, a second leg length L2 of 4.83 mm, and a throat thickness TH1 of 3.42 mm are obtained by visual inspection. Furthermore, the welding information acquisition device 20 according to this embodiment acquires a first leg length L1 of 5.42 mm, a second leg length L2 of 4.65 mm, and a throat thickness TH1 of 3.21 mm. Therefore, the difference between the measured values ​​of the first leg length L1 by visual inspection and those by the welding information acquisition device 20 is 0.23 mm, the difference between the measured values ​​of the second leg length L2 is 0.18 mm, and the difference between the measured values ​​of the throat thickness TH1 is 0.21 mm.

[0129] In the present embodiment, the throat thickness TH2 and the first and second leg lengths L3, L4 calculated by the leg length / throat thickness calculation unit 16 are based on the highest position (bead top) of the weld bead 110. However, as shown in Fig. 31 , the throat thickness TH2 and the first and second leg lengths L3, L4 calculated by the leg length / throat thickness calculation unit 16 may be based on the boundary line FL1 of the base material 100 estimated by the boundary line estimation unit 12.

[0130] In this embodiment, the cases where two base materials 100 form a T-joint (T-shaped joint) and a lap joint have been described, but the present invention can be applied not only to T-joints and lap joints, but also to cross joints and corner joints, as long as the base materials are welded at right angles and the configuration is fillet welded.

[0131] The welding information acquisition device disclosed herein can acquire information regarding the positions of two base materials to be welded, and is useful as a welding information acquisition device that acquires information regarding the welding state based on three-dimensional shape data.

[0132] 8 Memory unit (shape data memory unit) 20 Welding information acquisition device 28 Calculation unit 31 Three-dimensional shape measurement sensor 100 Base material 110 Weld bead D1, D2 Distance FL1 Boundary line FL2 Base material wire WL Width line

Claims

1. A welding information acquisition device comprising: a shape data storage unit that stores three-dimensional shape data obtained by scanning around a weld bead joining two base materials with a sensor, the data indicating the coordinates of multiple measurement points on the surface of the weld bead and the exposed surface of the base material; a calculation unit that generates bead area data that identifies the formation area of ​​the weld bead in a plane perpendicular to a predetermined direction based on the shape data; identifies, for each point group that has a common coordinate in a second direction perpendicular to both the predetermined direction and the first direction, a maximum change point at which the differential coefficient of the function is maximum when the coordinate of each measurement point in the predetermined direction is considered as a function of the coordinate in a first direction perpendicular to the predetermined direction; and estimates a boundary line between the two base materials based on the coordinates of multiple maximum change points that are outside the formation area of ​​the weld bead identified by the bead area data.

2. A welding information acquisition device as described in claim 1, wherein the plurality of measurement points are arranged at a first constant interval in the X-axis direction of an XYZ Cartesian coordinate system having the predetermined direction as the Z-axis direction and are arranged at a second constant interval in the Y-axis direction of the XYZ Cartesian coordinate system, and the calculation unit further acquires, based on the bead area data, an end point at which the direction of inclination of the contour of the weld bead changes in the opposite direction, identifies a first reference line and a plurality of second reference lines originating from the end point such that the first reference line and the plurality of second reference lines intersect within a predetermined angle range at a midpoint position of the second reference line, and identifies, among the X-axis direction and the Y-axis direction, a direction in which measurement points constituting a weld center line including at least a portion of the first reference line are distributed more narrowly as the first direction, and identifies a direction in which measurement points constituting the weld center line are distributed more widely as the second direction.

3. In the welding information acquisition device described in claim 2, the calculation unit calculates the distance between the intersection of the plurality of second reference lines with the boundary line and the intersection with the outer edge of the weld bead formation area identified by the bead area data.

4. A welding information acquisition device as described in claim 1, characterized in that the estimation of the boundary line is performed by the least squares method based on the coordinates of multiple maximum change points outside the formation area of ​​the weld bead identified by the bead area data.

5. The welding information acquisition device according to claim 1, wherein the predetermined direction is the direction in which the sensor is facing.

6. The welding information acquisition device according to claim 1, wherein the direction in which the sensor faces is inclined at the specified angle with respect to the specified direction, and the calculation unit corrects the shape data.

7. A welding information acquisition device comprising: an image data storage unit that stores three-dimensional shape data obtained by scanning around a weld bead joining two base materials with a sensor, the data indicating the coordinates of a plurality of measurement points on the surface of the weld bead and the exposed surface of the base material, the plurality of measurement points being arranged at a first constant interval in the X-axis direction of an XYZ orthogonal coordinate system having a predetermined direction as the Z-axis direction, and arranged at a second constant interval in the Y-axis direction of the XYZ orthogonal coordinate system; and a calculation unit that generates bead area data specifying a formation area of ​​the weld bead in a plane perpendicular to the predetermined direction based on the shape data, and estimates a base material line indicating the exposed surfaces of the two base materials in a cross section at the width line based on the Z coordinate of a measurement point that is outside the formation area of ​​the weld bead specified by the bead area data, has the same X coordinate as a component point that is not a measurement point, and is closest to the component point in the Y-axis direction.

8. The welding information acquisition device according to claim 7, wherein the calculation unit determines whether or not an intersection of the base metal wires of the two base materials exists in a position sandwiched between both ends of the width line in the width line direction in the cross section.

9. The welding information acquisition device according to claim 7, wherein the calculation unit estimates the base metal wire so that the base metal wire passes through the outer edge of the weld bead formation area identified by the bead area data.

10. A welding information acquisition device as described in claim 7, wherein the calculation unit further acquires an end point at which the direction of inclination of the contour of the weld bead changes in the opposite direction based on the bead area data, identifies a first reference line and a plurality of second reference lines originating from the end point such that the first reference line and the plurality of second reference lines each intersect within a predetermined angle range at a midpoint position of the second reference line, and identifies a straight line including the second reference line as the width line.

11. The welding information acquisition device according to claim 7, wherein the predetermined direction is the direction in which the sensor is facing.

12. The welding information acquisition device according to claim 7, wherein the direction in which the sensor faces is inclined at the specified angle with respect to the specified direction.

Images