Shape measuring device and shape measuring method

Abstract

To provide a shape measurement apparatus which can stably measure the shape dimension of a weld zone in a narrow part having a complicated shape.SOLUTION: A shape measurement apparatus comprises: an illumination unit which irradiates a weld bead of a weld zone with light; an imaging unit which captures an image of the weld bead; a drive unit which changes relative positions of the weld bead and the illumination unit and relative positions of the weld bead and the imaging unit; a calculation unit which extracts coordinates of a feature point in two or more images captured by making the relative positions with respect to the weld bead different by the imaging unit and calculates the position attitude of the imaging unit and three-dimensional coordinates of the weld bead; and a control unit which controls the drive unit according to the calculation result of the calculation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a shape measuring device and a shape measuring method. [Background technology]

[0002] In arc welding, it is important to control the amount of heat that the arc imparts to the base metal and form an appropriate bead shape. That is, by checking the shape of the weld bead, it is possible to confirm that the amount of heat received by the base metal is appropriate (within an expected range), thereby ensuring the quality of the arc welding. Known conventional technologies for measuring the shape of a weld bead include those described in Patent Documents 1 and 2, for example.

[0003] Patent Document 1 discloses a shape measurement system that includes a light source, an irradiation body that irradiates a target object with light from the light source, an imaging body that images the target object based on the light irradiated from the irradiation body to the target object, a movement mechanism that moves the irradiation body and imaging body along the target object, and a processing unit that executes data processing to measure the shape of the target object based on the image of the target object, wherein the imaging body captures images of the target object at predetermined intervals as it moves along the target object, and the processing unit determines the cross-sectional shape of the target object and the movement mode of the movement mechanism based on the image of the target object, and measures the shape of the target object based on the cross-sectional shape and the movement mode.

[0004] Patent Document 2 discloses a pipe inner surface shape measuring device that measures the inner surface shape of the pipe to be measured by inserting a detection head into the pipe to be measured, the detection head having two television cameras that photograph the inside of the pipe to be measured and a light projector that provides illumination when photographing with the television cameras, and the pipe inner surface shape measuring device is equipped with an image recording device that digitizes and records the video signals from the two television cameras at the same time, and a data processing device having a stereo method processing means that reads out the image data recorded in the image recording device and determines the inner surface shape of the pipe to be measured by a stereo method. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-179918 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-64589 Summary of the Invention [Problem to be solved by the invention]

[0006] In the conventional technology described in Patent Document 1, the cross-sectional shape of a measurement object is determined from an image acquired by irradiating the measurement object with laser light. However, because laser light is used for measurement, it is necessary to access the measurement object using a waveguide such as an optical fiber, and even if a waveguide with a small allowable bending radius is used, it is difficult to apply this technology to measurements of objects with complex shapes, such as curved piping.

[0007] Furthermore, in the prior art described in Patent Document 2, the inner shape of the pipe is determined using a stereo method, and it is necessary to adjust the camera posture and lighting position so that the feature points of the measurement object can be captured simultaneously with two television cameras. However, even if an attempt is made to simultaneously capture a weld bead with an uneven surface with two television cameras, adjusting the camera so that the feature points are captured by one camera does not necessarily mean that the feature points will be captured by the other camera, making stable shape measurement difficult.

[0008] The present invention has been made in view of the above, and aims to provide a shape measuring device and a shape measuring method that can stably measure the shape and dimensions of a weld in a narrow part having a complex shape. [Means for solving the problem]

[0009] The present application includes multiple means for solving the above-mentioned problems, and one example thereof includes an illumination unit that irradiates light onto a weld bead of a weld portion, an imaging unit that captures an image of the weld bead, a drive unit that changes the relative position of the weld bead to the illumination unit and the relative position of the weld bead to the imaging unit, a calculation unit that extracts coordinates of feature points in two or more images captured by the imaging unit at different relative positions to the weld bead and calculates the position and attitude of the imaging unit and the three-dimensional coordinates of the weld bead, and a control unit that controls the drive unit in accordance with the calculation results of the calculation unit. [Effects of the Invention]

[0010] According to the present invention, it is possible to stably measure the shape and dimensions of a weld in a narrow portion having a complex shape. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram schematically illustrating the overall configuration of a shape measuring device. [Figure 2] 10A and 10B are diagrams illustrating an example of adjusting the positions and orientations of the illumination unit and the image capture unit relative to the measurement target. [Figure 3] 10 is a flowchart showing the contents of a shape measurement process. [Figure 4] 10A and 10B are diagrams showing how a weld bead is photographed from various directions. [Figure 5] FIG. 1 is a diagram illustrating SfM. [Figure 6] FIG. 10 is a diagram showing a specific calculation of three-dimensional coordinates of a weld bead. [Figure 7] 10 is a flowchart showing a shape measurement process in the second embodiment. [Figure 8] 10A and 10B are diagrams illustrating how the position and orientation of the imaging unit are determined based on a three-dimensional point group and a normal vector. [Figure 9] 11 is a flowchart showing a shape measurement process in the third embodiment. [Figure 10] FIG. 1 is a diagram illustrating a Phong reflection model. [Figure 11] 13 is a flowchart showing a shape measurement process in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, when there are multiple identical components, they may be distinguished by adding an alphabet to the end of the same reference numeral (number), but the alphabet may be omitted and the multiple components may be collectively referred to. For example, when there are three normal vectors 15a, 15b, and 15c, these may be collectively referred to as normal vector 15. Furthermore, for simplicity, illustrations of signal lines and the like whose connection relationships are clear from the description may be omitted.

[0013] First Embodiment A first embodiment of the present invention will be described with reference to FIGS.

[0014] FIG. 1 is a diagram schematically showing the overall configuration of a shape measuring device according to the present embodiment.

[0015] In FIG. 1, the shape measuring device 100 is generally composed of an illumination unit 3, an imaging unit 4, a control unit 5, and a calculation unit 6.

[0016] In this embodiment, a weld bead 2 of a pipe 1 will be described as an example of a measurement target. The pipe 1 is formed by welding two cylindrical members of the same radius in the axial direction, and the weld bead 2 is formed in the welded portion around the circumference of the pipe 1. In the coordinate system 7 in FIG. 1, the x-axis is set in the longitudinal direction of the pipe 1, and the y-axis and z-axis are set perpendicular to each other in directions along a plane perpendicular to the x-axis. In particular, the imaging direction of the imaging unit 4 is set as the z-axis direction.

[0017] The lighting unit 3, which is an LED light source, emits light onto the weld bead 2 of the pipe 1, which is the measurement target. The LED light is connected to a power source (not shown), and the intensity of the emitted light can be adjusted by changing the voltage supplied from the power source to the lighting unit 3 in response to a command signal from the control unit 5.

[0018] The photographing unit 4 is, for example, a camera such as a videoscope or a fiberscope. A videoscope, shown as an example of the photographing unit 4 in this embodiment, has an objective lens and an image sensor installed at the tip of the scope, and can obtain an image signal of light received by the image sensor through the objective lens and transmit it to the control unit 5. The diameter of the videoscope is extremely thin, for example, a few millimeters, and it is possible to observe even a measurement object located in a narrow space.

[0019] The illumination unit 3 and the photographing unit 4 are arranged, for example, to extend in the x-axis direction along the central axis of the piping 1, and are supported by a moving mechanism 10a that adjusts the positions of the illumination unit 3 and the photographing unit 4 in the axial direction of the piping 1 (x-axis direction), and a rotation mechanism 10b that adjusts the circumferential positions of the illumination unit 3 and the photographing unit 4 relative to the x-axis (direction as viewed from the x-axis) by rotating the moving mechanism 10a in a direction around the axis.

[0020] Furthermore, the illumination unit 3 has a position adjustment mechanism 8a that adjusts the distance of the illumination unit 3 from the x-axis (the distance from the moving mechanism 10a in FIG. 1), and an angle adjustment mechanism 8b that adjusts the angle of light emitted by the illumination unit 3. Similarly, the photographing unit 4 has a position adjustment mechanism 9a that adjusts the distance of the photographing unit 4 from the x-axis (the distance from the moving mechanism 10a in FIG. 1), and an angle adjustment mechanism 9b that adjusts the photographing angle of the photographing unit 4. Note that the photographing unit 4 may be provided with a tip movable mechanism that can change the angle of its tip by 360°.

[0021] The position adjustment mechanisms 8a and 9a and the movement mechanism 10a are automatic stages that translate the stage position using, for example, a stepping motor or a servo motor, while the angle adjustment mechanisms 8b and 9b and the rotation mechanism 10b are automatic stages that rotate the stage position using, for example, a stepping motor or a servo motor.

[0022] The positions and directions of the illumination unit 3 and the imaging unit 4 can be freely adjusted by position adjustment mechanisms 8a and 9a, a movement mechanism 10a, angle adjustment mechanisms 8b and 9b, and a rotation mechanism 10b.

[0023] Illumination unit 3 and photography unit 4 are moved in the x-axis direction (the axial direction of pipe 1) by movement mechanism 10a, and the distance to the inner surface of pipe 1 is adjusted by position adjustment mechanisms 8a and 9a, thereby adjusting the position near weld bead 2 on pipe 1. Then, by rotating movement mechanism 10a about its axis by rotation mechanism 10b, illumination unit 3 and photography unit 4 are moved in the circumferential direction of the x-axis around the x-axis along weld bead 2 around the entire circumference of pipe 1, and can photograph weld bead 2. At this time, illumination unit 3 and photography unit 4 can photograph weld bead 2 while adjusting the light irradiation angle and photography angle relative to weld bead 2 by angle adjustment mechanisms 8b and 9b.

[0024] The control unit 5 realizes its functions using a controller and a program executed by a general computer, and controls the operations of the illumination unit 3, the photographing unit 4, the position adjustment mechanisms 8a and 9a, the movement mechanism 10a, the angle adjustment mechanisms 8b and 9b, and the rotation mechanism 10b based on command signals from the calculation unit 6.

[0025] The functions of calculation unit 6 are realized by a program executed by a general computer. Calculation unit 6 extracts coordinates of feature points of weld bead 2 from two or more images of weld bead 2 captured by imaging unit 4, and calculates the position and orientation of imaging unit 4 and a three-dimensional point cloud (described later) of weld bead 2.

[0026] Although not shown, the computer that controls the control unit 5 and the calculation unit 6 includes a central control unit, input units such as a mouse and a keyboard, a display unit such as a display, a storage unit such as a memory and a hard disk, and a communication unit. The functions of the control unit 5 and the calculation unit 6 are realized by executing a program loaded into the storage unit.

[0027] 2 is a diagram showing an example of adjusting the position and orientation of the illumination unit and the imaging unit relative to the measurement target. In FIG. 2, the pipe 1, the weld bead 2, and the shape measuring device 100 are shown as viewed from the x-axis direction (the axial direction of the pipe 1) in FIG.

[0028] 2, photographing unit 4 acquires images of weld bead 2 of piping 1. At this time, photographing unit 4 adjusts its position in a direction along the yz plane (the distance from weld bead 2 to photographing range 11) using position adjustment mechanism 9a, and adjusts its angle around the x-axis (the angle of photographing range 11 relative to weld bead 2) using angle adjustment mechanism 9b. This allows photographing unit 4 to acquire images of weld bead 2 from various angles while maintaining a photographing magnification suitable for photographing weld bead 2.

[0029] Furthermore, illumination unit 3 irradiates light near imaging range 11 of imaging unit 4. At this time, illumination unit 3 adjusts its position in a direction along the yz plane (the position of imaging range 11 relative to weld bead 2) using position adjustment mechanism 8a, and adjusts its angle around the x-axis (the angle of imaging range 11 relative to weld bead 2) using angle adjustment mechanism 8b. This allows the position and orientation of illumination unit 3 to be set so that the brightness is suitable for imaging weld bead 2.

[0030] FIG. 3 is a flowchart showing the details of the shape measurement process.

[0031] In FIG. 3, first, the shape measuring device 100 moves the illumination unit 3 and the photographing unit 4 to a measurement position of the weld bead 2 of the pipe 1 that is the measurement target (step S101).

[0032] At the measurement position, while checking the image acquired by the photographing unit 4, the position and orientation of the illumination unit 3 are adjusted so that the weld bead 2 can be clearly observed. Because the weld bead 2 basically has a convex shape and a metallic luster, the appearance of the image changes sensitively depending on the intensity and position and orientation of the illumination unit 3. Therefore, the intensity and position and orientation of the illumination unit 3 are adjusted so that the pattern of the weld bead 2 can be seen accurately, that is, so that the desired image can be captured by the photographing unit 4.

[0033] The movement of the illumination unit 3 and the photographing unit 4 to the measurement position and the adjustment of their position and orientation may be performed by specifying a specific position in advance using a program, or may be performed manually while checking the image acquired by the photographing unit 4.

[0034] Next, the photographing unit 4 acquires an image of the photographing range 11 of the weld bead 2 (step S102). The image photographed by the photographing unit 4 is transferred to a computer connected to the photographing unit 4 and stored in a storage device (not shown).

[0035] The processing of steps S101 and S102 is repeated while changing the position and angle of the image capturing unit 4 relative to the weld bead 2, and images of the weld bead 2 captured from various angles are obtained.

[0036] FIG. 4 shows how the weld bead is photographed from various directions.

[0037] Because weld bead 2 basically has a convex shape, as shown in Fig. 4, the position and orientation of photographing unit 4 is adjusted so that the convex surface and photographing unit 4 face each other, that is, so that photographing unit 4 photographs from the normal side of the surface of weld bead 2. By positioning photographing unit 4 so that it faces the surface of weld bead 2 and acquiring multiple images while maintaining a constant distance between photographing unit 4 and weld bead 2, distortion of the captured images is reduced and the magnification of the captured images becomes approximately equal. This reduces errors in matching feature points, which will be described later, and allows for highly accurate shape measurement of weld bead 2.

[0038] After the process of step S102 is completed, feature points are extracted from the multiple images of the weld bead 2 acquired in step S102 (step S103). The feature to be extracted is, for example, SIFT (Scale-Invariant Feature Transform). SIFT is known as a highly robust feature because it is invariant to scale changes caused by enlargement or reduction.

[0039] Next, the feature amounts extracted in step S103 are compared with feature points extracted from multiple images acquired with different positions and orientations of the image capture unit 4 relative to the weld bead 2, and the feature points in the multiple images are matched (step S104). If a sufficient number of feature amounts are similar between multiple image pairs, they can be considered to correspond to each other.

[0040] Next, based on the relationship between the feature points associated in step S104, the posture of the imaging unit 4 and the three-dimensional coordinates of the weld bead 2 are calculated by bundle adjustment (step S105). For the bundle adjustment, for example, an SfM (Structure from Motion) algorithm is used.

[0041] FIG. 5 is a diagram illustrating SfM.

[0042] As shown in Fig. 5, in SfM, the three-dimensional coordinate X of the measurement object is estimated from m images taken by changing the camera's orientation relative to the measurement object. Here, the three-dimensional coordinate X projected onto the i-th image (i = 1, , m) is estimated to satisfy the following (Equation 1).

[0043]

number

[0044] In the above formula (1), λ is a scaling factor, which is a coefficient that converts the pixel unit of the image into a length unit. λ can be uniquely determined if the distance between two specific cameras or the distance between two specific three-dimensional coordinates is known.

[0045] Furthermore, P in the above (Equation 1) is a camera matrix, and is expressed by the following (Equation 2).

[0046]

number

[0047] As shown in the above (Equation 2), the camera matrix K is composed of the camera's focal length f, the position of the optical center (cx, cy), the shear coefficient s that represents the magnitude of image distortion, and the anisotropy constant a, which are called intrinsic parameters. Furthermore, R and T are the rotation matrix and translation vector that represent the camera's orientation, which are called extrinsic parameters.

[0048] As shown in Figure 5, there is an error ei between the projected coordinates (estimated coordinates) of the three-dimensional coordinate X onto the i-th image estimated using the above (Equation 1) and the projected coordinates (observed coordinates) of the three-dimensional coordinate X of the measurement object observed on the i-th image. In bundle adjustment, the camera matrix P and the three-dimensional coordinate X are determined by minimizing the error ei. Using the sum of the squared distances of ei as a measure of the error, the reprojection error E is defined as shown in the following (Equation 3).

[0049]

number

[0050] In this case, bundle adjustment is reduced to the problem of minimizing the reprojection error E shown in Equation 3 above. A least-squares algorithm such as the Gauss-Newton method or the Levenberg-Macart method is generally used to minimize the reprojection error E. In other words, by updating the solution using the least-squares algorithm and performing calculations until the change in the reprojection error E or the change in the three-dimensional coordinate X converges, the three-dimensional coordinate X of the measurement object can be obtained.

[0051] FIG. 6 is a diagram showing a specific calculation of the three-dimensional coordinates of the weld bead.

[0052] As shown in Figure 6, by performing bundle adjustment using three images 40a, 40b, and 40c taken with the photographing unit 4 at different angles (postures) relative to the convex shape of the weld bead 2, information on the actual postures 4a, 4b, and 4c of the photographing unit 4 at the time of photographing and three-dimensional point cloud information of the weld bead 2 can be obtained.

[0053] The effects of the present embodiment configured as above will be described.

[0054] Generally, the SfM algorithm used in bundle adjustment requires the following conditions (1) to (3) to accurately estimate three-dimensional coordinates. (1) A large number of feature points can be extracted from images of the object being measured. (2) There are few areas in the image where brightness changes significantly. (3) Use images in which the camera position and orientation are changed slightly.

[0055] The weld bead satisfies the condition (1) above because a characteristic pattern and surface irregularities are formed when the molten metal solidifies. In this invention, we focused on the characteristics of the weld bead as described above and conceived the application of SfM.

[0056] On the other hand, to satisfy condition (2) above for weld beads with surface irregularities and metallic luster, it is necessary to adjust the lighting conditions. Therefore, the shape measurement device of the present invention is designed to be able to change the relative position and posture between the weld bead and the lighting unit, or between the weld bead and the imaging unit, so that feature points can be stably captured even for weld beads with surface irregularities. This satisfies not only condition (2) above, but also condition (3), enabling stable and highly accurate shape measurement of weld beads using SfM.

[0057] Furthermore, in this embodiment, a driving unit is provided to change the relative positions of an illumination unit that irradiates the weld bead with light and an imaging unit that captures an image of the weld bead relative to the weld bead, and the shape of the weld bead is measured using SfM. This allows for stable measurement of the shape and dimensions of a weld in a narrow, complex shape. In other words, because a camera with a large allowable bending radius, such as a videoscope, is used, this method can also be applied to welds in narrow, complex shapes, such as bent pipes. Furthermore, because two or more images are used to perform bundle adjustment to minimize reconstruction error, the three-dimensional coordinates of the weld bead can be calculated with high accuracy.

[0058] <Second embodiment> A second embodiment of the present invention will be described with reference to FIGS.

[0059] In this embodiment, the position and orientation of the imaging unit are determined in addition to the processing of the first embodiment. In this embodiment, the same reference numerals are used for the same parts as in the first embodiment, and the description thereof will be omitted.

[0060] Fig. 7 is a flowchart showing the shape measurement process in this embodiment, and Fig. 8 is a diagram showing how the position and orientation of the imaging unit are determined based on a three-dimensional point group and a normal vector.

[0061] 7, the profile measuring device 100 first performs the processes of steps S101 to S105. The processes of steps S101 to S105 are the same as the processes described in FIG. 3 of the first embodiment, and therefore the description thereof will be omitted.

[0062] After completing the process of step S105, the shape measuring device 100 subsequently calculates the normal vector 15 from the three-dimensional point cloud 14 (step S106). Known methods for calculating the normal vector 15 include, for example, a method using the cross product of vectors or principal component analysis.

[0063] In the method using the cross product of vectors, a triangular mesh is generated from the three-dimensional point group 14, and the normal vector 15 can be calculated by calculating the cross product of vectors connecting three vertices of the mesh.

[0064] In addition, in a method using principal component analysis, a set of k points (k-neighborhood) selected in order of proximity to a point pi in the three-dimensional point cloud 14 is extracted, and a covariance matrix C defined by the following (Equation 4) is calculated.

[0065]

number

[0066] Here, in the above (Equation 4), pc represents the coordinates of the center of gravity of the group of neighboring points.

[0067] Next, the covariance matrix C is diagonalized to obtain the eigenvalues ​​and eigenvectors expressed by the relationship in the following (Equation 5).

[0068]

number

[0069] Here, in the above (Equation 5), vj is the j-th eigenvector, λj is the j-th eigenvalue, and the order j is defined as 1st, 2nd, 3rd in descending order of eigenvalue.

[0070] In this case, the directions of the first and second eigenvectors can be considered to be the directions with the most points and the second most points, and the plane formed by these two directions is considered to be the plane that best fits the k-nearest neighbors. Therefore, the third eigenvector is estimated to be the normal vector at point pi.

[0071] After the process in step S106 is completed, the position and orientation of the image capturing unit 4 is determined based on the normal vector 15 calculated in the process in step S106 (step S107).

[0072] 8, when determining the position and orientation of the imaging unit 4 based on the normal vectors 15a to 15c of the three-dimensional point cloud 14, for example, a virtual imaging unit 16a is placed at a point obtained by moving point xi of the three-dimensional point cloud 14 by a certain distance h in the direction of normal vector 15a(n(xi)), and the virtual imaging unit 16 is directed in the negative direction of normal vector 15a(n(xi)). The position and orientation of this virtual imaging unit 16a are then determined as the position and orientation of the imaging unit 4. Furthermore, using a similar procedure, the positions and orientations of virtual imaging units 16b and 16c are determined as the positions and orientations of the imaging unit 4 for each of the other points xj and xk of the three-dimensional point cloud 14.

[0073] The other configurations are the same as those in the first embodiment.

[0074] The present embodiment configured as above can also achieve the same effects as the first embodiment.

[0075] Furthermore, since the position and orientation of the imaging unit 4 are automatically determined based on the three-dimensional point cloud 14, the time and effort required for manual adjustment while checking the images acquired by the imaging unit 4 can be eliminated, thereby shortening the measurement time.

[0076] Furthermore, the imaging unit 4 can be positioned so as to face the weld bead 2, and the distance between the weld bead 2 and the imaging unit can be kept constant. This makes it easier to obtain good results in matching feature points, improving the accuracy of three-dimensional reconstruction and enabling the shape of the weld bead to be measured with higher precision.

[0077] <Third embodiment> A third embodiment of the present invention will be described with reference to FIGS.

[0078] In this embodiment, in addition to the processing of the second embodiment, calculation of image brightness and determination of lighting conditions are performed. In this embodiment, the same reference numerals are used for the same parts as in the first and second embodiments, and the description thereof will be omitted.

[0079] FIG. 9 is a flowchart showing the shape measurement process in this embodiment.

[0080] 9, the profile measuring device 100 first performs the processes of steps S101 to S107. The processes of steps S101 to S107 are the same as the processes described in FIG. 3 of the first embodiment and the processes described in FIG. 7 of the second embodiment, and therefore, description thereof will be omitted.

[0081] After completing the process of step S107, the form measuring device 100 subsequently calculates the brightness of the image using the normal vector 15 of the three-dimensional point cloud 14 calculated in the process of step S106 (step S108). The brightness of the image is calculated using, for example, the Phong reflection model.

[0082] FIG. 10 is a diagram illustrating the Phong reflection model.

[0083] In the Phong reflection model, the components of light reflected from the surface of an object are considered to be divided into an ambient reflection component i_i, a diffuse reflection component i_i, and a specular reflection component i_i. Assuming a virtual lighting unit 17, light incident on point xi in the 3D point cloud 14 from the virtual lighting unit 17 is observed as a diffuse reflection component i_d and a specular reflection component i_s. When ambient light is present, the ambient reflection component i_a is also superimposed. The intensity of the diffuse reflection component i_i detected by the virtual imaging unit 16 varies depending on the angle α between the direction from the virtual lighting unit 17 toward point xi and the normal vector 15 at point xi. Meanwhile, the intensity of the specular reflection component is detected by the virtual imaging unit 106 varies depending on the angle β between the line of sight 19 and the light ray 18 completely reflected at point xi from the virtual lighting unit 17. The light intensity Ip(xi) observed by the virtual imaging unit 16 is expressed by the following equation (6):

[0084]

number

[0085] Here, in the above (Equation 6), ka is the environmental reflection coefficient, kd is the diffuse reflection coefficient, ks is the specular reflection coefficient, and n is the glossiness, which are constants determined by the material.

[0086] Using the above (Equation 6), the brightness of the image observed by the virtual photographing unit 16 when the position and orientation of the virtual illumination unit 17 is changed with respect to the three-dimensional point cloud 14 is calculated.

[0087] After the process of step S108 is completed, the intensity or position and orientation of the illumination unit 3 is determined based on the image brightness calculated in step S108 (step S109). Because the weld bead 2 basically has a convex shape and metallic luster, the appearance of the image changes sensitively depending on the intensity and position and orientation of the illumination unit 3. Furthermore, if the intensity of the illumination unit 3 is too high, halation occurs due to the metallic luster of the weld bead 2, resulting in saturation of the image brightness. On the other hand, if the intensity of the illumination unit 3 is low, the entire image becomes dark, making it difficult to detect the pattern of the weld bead 2. Therefore, as in the process of step S108, an appropriate position and orientation of the virtual illumination unit 17 is selected based on the calculated image brightness by changing the position and orientation of the virtual illumination unit 17, and this is determined as the position and orientation of the illumination unit 3. For example, the position and orientation of the virtual illumination unit 17 whose brightness is within a certain range for all pixels of the image and whose average brightness is the largest is selected as the position and orientation of the illumination unit 3. This allows the position and orientation of the illumination unit 3 to be determined so that the image captured by the image capture unit 4 at the position of the virtual capture unit 16 has appropriate brightness and is easy to observe overall.

[0088] The other configurations are the same as those of the first and second embodiments.

[0089] The present embodiment configured as above can also achieve the same effects as the first and second embodiments.

[0090] Furthermore, because the position and orientation of the illumination unit 3 can be automatically determined based on the calculated brightness of the image, it is possible to omit the process of manually adjusting the position and orientation of the illumination unit 3 while checking the image from the imaging unit 4, thereby shortening the measurement time. Furthermore, by setting appropriate illumination conditions, it is possible to improve the density and accuracy of points in three-dimensional reconstruction, enabling the shape of the weld bead to be measured more precisely and with higher accuracy.

[0091] <Fourth embodiment> A fourth embodiment of the present invention will be described with reference to FIG.

[0092] In this embodiment, in addition to the processing of the first embodiment, the calculation of image brightness and the determination of lighting conditions are performed as explained in the third embodiment. In this embodiment, the same reference numerals are used for the same parts as those in the first and third embodiments, and the explanation thereof will be omitted.

[0093] FIG. 11 is a flowchart showing the shape measurement process in this embodiment.

[0094] 11, the profile measuring device 100 first performs the processes of steps S101 to S105. The processes of steps S101 to S105 are the same as the processes described in FIG. 3 of the first embodiment, and therefore the description thereof will be omitted.

[0095] After completing the process of step S105, the shape measuring device 100 subsequently calculates the normal vector 15 of the three-dimensional point cloud 14, and calculates the brightness of the image using this normal vector 15 (step S108). The normal vector 15 can be calculated, for example, by using the cross product of vectors or a method using principal component analysis (see the second embodiment).

[0096] After the process of step S108 is completed, the intensity or the position and orientation of the illumination unit 3 is subsequently determined based on the luminance of the image calculated in step S108 (step S109).

[0097] The other configurations are the same as those of the first and third embodiments.

[0098] The present embodiment configured as above can also achieve the same effects as the first and second embodiments.

[0099] <Additional Notes> The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0100] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-described configurations, functions, etc. may be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or a solid-state drive (SSD), or a recording medium such as an integrated circuit (IC) card, an SD card, or a digital versatile disc (DVD).

[0101] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it is considered that almost all components are interconnected. [Explanation of symbols]

[0102] 1...piping, 2...weld bead, 3...lighting unit, 4...photography unit, 5...control unit, 6...calculation unit, 7...coordinate system, 8a, 9a...position adjustment mechanism, 8b, 9b...angle adjustment mechanism, 10a...movement mechanism, 10b...rotation mechanism, 11...photography range, 14...three-dimensional point cloud, 15...normal vector, 16...virtual photography unit, 17...virtual lighting unit, 18...light ray, 19...line of sight direction, 100...shape measurement device

Claims

1. an illumination unit that irradiates light onto the weld bead of the weld; an imaging unit that captures an image of the weld bead; a driving unit that changes the relative position between the weld bead and the lighting unit, and the relative position between the weld bead and the photographing unit; a calculation unit that extracts coordinates of feature points in two or more images captured by the imaging unit at different relative positions with respect to the weld bead, and calculates the position and orientation of the imaging unit and three-dimensional coordinates of the weld bead; a control unit that controls the drive unit in accordance with the calculation result of the calculation unit; A device for measuring the shape of a weld, comprising:

2. 2. The shape measuring device according to claim 1, The calculation unit Calculating a normal direction to a surface of the weld bead from the three-dimensional coordinates of the weld bead; Calculating the position and orientation of the lighting unit or the imaging unit based on the normal direction A shape measuring device characterized by:

3. 3. The shape measuring apparatus according to claim 2, The calculation unit calculating a brightness of the image of the weld bead based on the position of the illumination unit and a normal direction to the surface of the weld bead; Calculating the position and orientation of the lighting unit based on the luminance A shape measuring device characterized by:

4. a step of irradiating a weld bead of a welded portion with light from an illumination unit while capturing an image of the weld bead with a photographing unit; extracting coordinates of feature points in two or more images captured at different relative positions with respect to the weld bead, and calculating the position and orientation of the image capturing unit and three-dimensional coordinates of the weld bead; changing a relative position between the weld bead and the lighting unit and a relative position between the weld bead and the photographing unit according to the calculated three-dimensional coordinates of the weld bead; A method for measuring the shape of a weld, comprising:

5. 5. The shape measuring method according to claim 4, calculating a normal direction to a surface of the weld bead from the three-dimensional coordinates of the weld bead; calculating the position and orientation of the illumination unit or the image capture unit based on the normal direction; A shape measuring method comprising:

6. 6. The shape measuring method according to claim 5, calculating the brightness of the image of the weld bead from the position of the illumination unit and a normal direction to the surface of the weld bead; calculating the position and orientation of the illumination unit based on the luminance; A shape measuring method comprising:

Images