Appearance inspection device, welding system, and appearance inspection method for welding site

Abstract

The appearance inspection device (20) is provided with: a shape measurement unit (21) that measures the three-dimensional shape of the welding site (201) in the Y direction; a shape data processing unit (24) that reconstructs the shape data; and a first determination unit (28) that determines whether or not the shape of the welded portion (201) is good on the basis of the reconstructed shape data and the determination model. When the welding site (201) includes a curved portion, the shape data processing unit (24) corrects the coordinate points of the shape data located at different positions in the X-direction so that the Y-direction resolutions of the shape data are the same, and determines the shape data on the basis of the coordinate position in the X-direction and the coordinate position in the Y-direction after the resolution correction. The shape data is reconstructed by correcting the coordinate position of the shape data in the Z direction.

Description

Technical Field

[0001] The present disclosure relates to an appearance inspection device, a welding system, and a method for inspecting the appearance of a welded part. Background Art

[0002] In recent years, image recognition technology for detecting specific objects from image data using a detection model strengthened by machine learning has been popularized. In this case, the detection model is learned using pre-prepared learning data, but sometimes the learning data contains data unsuitable for learning the detection model. Therefore, techniques for appropriately excluding unsuitable data to improve the determination accuracy of the detection model have been proposed (for example, refer to Patent Document 1).

[0003] In addition, techniques for performing appearance inspection of a welded part or the like using a determination model strengthened by machine learning and determining the quality of the shape have also been proposed.

[0004] For example, Patent Document 2 proposes an appearance inspection device for a welded part, which includes a shape measurement unit, an image processing unit, a learning data set generation unit, a determination model generation unit, and a first determination unit.

[0005] The shape measurement unit measures the shape of the welded part, and the image processing unit generates image data of the welded part based on the measured shape data. The learning data set generation unit classifies a plurality of image data according to the material and shape of each workpiece, and performs data augmentation to generate a plurality of learning data sets. The determination model generation unit uses the plurality of learning data sets to generate a determination model for the shape of the welded part according to the material and shape of each workpiece. The first determination unit determines the quality of the shape of the welded part based on the image data read from the image processing unit and the determination model.

[0006] Patent Document 1: International Publication No. 2019 / 187594

[0007] Patent Document 2: International Publication No. 2020 / 129617 Summary of the Invention

[0008] - Technical Problem to be Solved by an Invention

[0009] Generally, in the case of strengthening a determination model for determining the quality of the shape of a welded part by learning, the prepared learning data is generated based on a small number of types of shape data. Typically, this shape data is the shape data of the welded part obtained under one inspection condition according to the material and shape of each workpiece. This is because preparing learning data for each shape of the welded part requires a large amount of man-hours and costs.

[0010] For example, the shape of a weld formed in a straight line is measured, and learning data is prepared using the obtained shape data. Additionally, typically, when measuring the shape of a weld, shape measurement is performed while moving a sensor along the length direction of the weld at a constant speed. Therefore, in the learning data, the measurement resolution along the length direction of the weld is mostly the same regardless of the position in the width direction of the weld.

[0011] However, in actual workpieces, the shapes of welds as welding parts are diverse. For example, there are welds formed in a straight line and welds formed in a curved shape. In a curved weld, the moving distance of the sensor at each prescribed time interval varies according to the position in the width direction of the weld. In this case, the measurement resolution along the length direction of the weld changes according to the position in the width direction of the weld.

[0012] Therefore, when inputting the shape data of a curved weld into a determination model, the characteristics of the shape determined from the learning data often do not match the characteristics of the actual weld shape. In this case, even when using a determination model with completed learning reinforcement, it may not be possible to accurately determine the quality of the shape of the welding part.

[0013] The present disclosure has been completed to solve the above technical problems, and its object is to provide an appearance inspection device, a welding system, and an appearance inspection method for a welding part that can accurately evaluate the three-dimensional shape of a welding part even when the welding part includes a curved portion.

[0014] -Technical solutions for solving technical problems-

[0015] To achieve the above object, the appearance inspection device according to the present disclosure is an appearance inspection device for inspecting the appearance of the welded portion of a workpiece, and is characterized in that: the appearance inspection device at least includes a shape measurement unit and a data processing unit. The shape measurement unit is mounted on a robot and measures the three-dimensional shape of the welded portion along a welding line. The data processing unit processes the shape data obtained by the shape measurement unit. The data processing unit at least includes a shape data processing unit and a first determination unit. The shape data processing unit reconstructs the shape data obtained by the shape measurement unit. The first determination unit determines whether the shape of the welded portion is good or not based on the shape data reconstructed by the shape data processing unit and one or more determination models prepared in advance. In the plane where the welded portion is formed, when the direction orthogonal to the welding line is set as the X direction, the direction extending along the welding line is set as the Y direction, and the direction orthogonal to the X direction and the Y direction respectively is set as the Z direction, the shape measurement unit measures the shape of the welded portion at a plurality of points along the X direction and measures the shape of the welded portion at a plurality of points along the Y direction. When the welded portion includes a curved portion, the shape data processing unit corrects the coordinate points of the shape data located at different positions along the X direction so that the Y-direction resolution of the shape data is the same, and corrects the coordinate position in the Z direction of the shape data based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, thereby reconstructing the shape data.

[0016] The welding system according to the present disclosure is characterized in that: the welding system includes the appearance inspection device and a welding device for welding the workpiece. The welding device at least includes a welding head and an output control unit. The welding head is used to input heat to the workpiece, and the output control unit controls the welding output of the welding head.

[0017] The method for inspecting the appearance of a welded part according to the present disclosure is a method for inspecting the appearance of a welded part using the said appearance inspection device, characterized in that: the method for inspecting the appearance of the welded part at least includes: a first step, in which the shape measurement unit measures the three-dimensional shape of the welded part while moving together with the robot and obtains the shape data; a second step, in which the shape data processing unit reconstructs the shape data; and a third step, in which the first determination unit determines whether the shape of the welded part is good or not based on the shape data reconstructed by the shape data processing unit and one or more determination models prepared in advance. In the second step, when the welded part includes a curved portion, the shape data processing unit corrects the coordinate points of the shape data located at different positions along the X direction so that the Y direction resolution of the shape data is the same, and based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, corrects the coordinate position in the Z direction of the shape data, thereby reconstructing the shape data.

[0018] - Effects of the Invention -

[0019] According to the present disclosure, it is possible to accurately evaluate the three-dimensional shape of a welded part with good accuracy regardless of the shape of the welded part. In addition, it is possible to accurately determine whether the shape of the welded part is good or not. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram showing the structure of the welding system according to the first embodiment;

[0021] Figure 2 is a schematic diagram showing the hardware structure of the robot control unit;

[0022] Figure 3 is a functional block diagram of the appearance inspection device;

[0023] Figure 4 is a schematic diagram showing the situation where the shape of the weld seam is measured by the shape measurement unit;

[0024] Figure 5A is a schematic diagram showing the hardware structure of the sensor control unit;

[0025] Figure 5B is a schematic diagram showing the hardware structure of the data processing unit;

[0026] Figure 6A is a top view schematic diagram showing an example of a shape defect mode of a welded part;

[0027] Figure 6B is along Figure 6ASchematic cross-sectional view taken along the VIB-VIB line;

[0028] Figure 6C is along Figure 6A Schematic cross-sectional view taken along the VIC-VIC line;

[0029] Figure 6D is along Figure 6A Schematic cross-sectional view taken along the VID-VID line;

[0030] Figure 6E is along Figure 6A Schematic cross-sectional view taken along the VIE-VIE line;

[0031] Figure 7 Flowchart of the appearance inspection steps for the welding part;

[0032] Figure 8A Schematic diagram showing the measurement point positions of the shape data before reconstruction;

[0033] Figure 8B Schematic diagram showing the measurement point positions of the shape data after reconstruction;

[0034] Figure 9 Flowchart showing the reconstruction steps of the shape data;

[0035] Figure 10 Schematic diagram showing the relationship between the Y-direction resolution and circumferential speed at the curved part of the welding part, and the distance from the center of the curved part along the X-direction to the measurement point;

[0036] Figure 11 Schematic diagram showing the relationship between the measurement point positions at the curved part of the welding part and various parameters;

[0037] Figure 12A Schematic diagram showing the three-dimensional coordinate positions of the measurement points before and after reconstructing the shape data;

[0038] Figure 12B Viewed from the Z direction Figure 12A when;

[0039] Figure 13 Conceptual diagram showing the derivation steps of the coordinate points of the shape data based on the bilinear interpolation method;

[0040] Figure 14A Stereogram of the workpiece related to Example 1;

[0041] Figure 14B Viewed from the Z direction Figure 14A when;

[0042] Figure 15AIt is a schematic diagram showing the measurement point positions of the shape data before reconstruction;

[0043] Figure 15B It is a schematic diagram showing the measurement point positions of the shape data after reconstruction;

[0044] Figure 16 It is a perspective view of the workpiece involved in Example 2;

[0045] Figure 17 It is a schematic diagram showing the measurement steps of the welding part in the conventional method;

[0046] Figure 18 It is a schematic diagram showing the measurement steps of the welding part in Example 2;

[0047] Figure 19 It is a perspective view of the workpiece involved in Example 3;

[0048] Figure 20 It is a functional block diagram of the appearance inspection device involved in the second embodiment;

[0049] Figure 21 It is a flowchart of the appearance inspection steps of the welding part involved in the second embodiment;

[0050] Figure 22A It is a schematic diagram showing an example of the production steps of the learning data set;

[0051] Figure 22B It is a schematic diagram showing another example of the production steps of the learning data set;

[0052] Figure 22C It is a schematic diagram showing still another example of the production steps of the learning data set. Detailed Embodiments

[0053] Hereinafter, embodiments of the present disclosure will be described based on the drawings. It should be noted that the following preferred embodiments are merely examples for essentially explaining the present disclosure, and are not intended to limit the present disclosure, its application objects, or its uses.

[0054] (First Embodiment)

[0055] [Structure of Welding System]

[0056] Figure 1 It is a schematic diagram showing the structure of the welding system according to the first embodiment. The welding system 100 includes a welding device 10 and an appearance inspection device 20.

[0057] The welding device 10 includes a welding torch 11, a wire feeding device 13, a power source 14, an output control unit 15, a robot 16, and a robot control unit 17. By supplying power from the power source 14 to the welding wire 12 held by the welding torch 11, an electric arc is generated between the tip of the welding wire 12 and the workpiece 200, and the workpiece 200 is input with heat to perform arc welding. It should be noted that the welding device 10 has other components and equipment such as a pipeline for supplying a shielding gas to the welding torch 11 and a gas cylinder, but for the sake of simplicity of explanation, their illustration and description are omitted. It should be noted that the power source 14 is also referred to as a welding power source.

[0058] The output control unit 15 is connected to the power source 14 and the wire feeding device 13. The output control unit 15 controls the welding output of the welding torch 11 according to specified welding conditions. In other words, it controls the power supplied to the welding wire 12 and the power supply time. In addition, the output control unit 15 controls the feeding speed and feeding amount of the welding wire 12 fed from the wire feeding device 13 to the welding torch 11. It should be noted that regarding the welding conditions, they can be directly input into the output control unit 15 via an input unit (not shown), or a welding program read from another recording medium or the like can be selected.

[0059] The robot 16 is a well-known multi-joint axis robot that holds the welding torch 11 at its tip and is connected to the robot control unit 17. The robot control unit 17 controls the movement of the robot 16 so that the tip of the welding torch 11, in other words, the tip of the welding wire 12 held by the welding torch 11, traces a specified welding trajectory and moves to a desired position.

[0060] Figure 2 A schematic diagram showing the hardware structure of the robot control unit. The robot control unit 17 includes at least a CPU (Central Processing Unit) 17a, a driver IC (Driver Integrated Circuit) 17b, a RAM (Random Access Memory) 17c, and an IC (Integrated Circuit) 17d.

[0061] In normal operation, the IC 17d receives the output signals of rotation detectors (not shown) respectively provided on a plurality of joint axes, and the above-mentioned plurality of joint axes are provided on the robot 16. After the output signals are processed by the IC 17d, they are input into the CPU 17a. The CPU 17a sends a control signal to the driver IC 17b based on the signals input from the IC 17d and the rotational speeds of the joint axes set in a specified program stored in the RAM 17c. The driver IC 17b controls the rotation of a servo motor (not shown) connected to the joint axis based on the control signal from the CPU 17a.

[0062] In addition, as described later, in the appearance inspection device 20, when the first determination unit 28 of the data processing unit 23 determines that the shape of the welded part 201 is defective, the CPU 17a of the robot control unit 17 that has received this determination result stops the operation of the robot 16, or operates the robot 16 in such a way that the torch 11 comes to a specified initial position.

[0063] In addition, the output control unit 15 also has the same structure as the robot control unit 17. That is to say, the output control unit 15 at least includes a CPU 15a, a driving IC 15b, a RAM 15c, and an IC 15d.

[0064] In normal operation, the IC 15d receives a signal corresponding to the output of the power supply 14. After this signal is processed by the IC 15d, it is input into the CPU 15a. The CPU 15a sends a control signal to the driving IC 15b based on the signal input from the IC 15d and the output of the power supply 14 set in a specified program stored in the RAM 15c. The driving IC 15b controls the output of the power supply 14 based on the control signal from the CPU 15a, and further controls the welding output of the torch 11.

[0065] In addition, as described later, in the appearance inspection device 20, when the first determination unit 28 determines that the shape of the welded part 201 is defective, the CPU 15a of the output control unit 15 that has received this determination result stops the output of the power supply 14. Thereby, the welding output of the torch 11 is stopped.

[0066] It should be noted that both the output control unit 15 and the robot control unit 17 may include Figure 2 components other than those shown. For example, it may also include a ROM (Read Only Memory) and an HDD (Hard Disk Drive) as storage devices.

[0067] The appearance inspection device 20 has a shape measurement unit 21, a sensor control unit 22, and a data processing unit 23. The shape measurement unit 21 is installed on the robot 16 or the torch 11 and measures the shape of the welded part 201 of the workpiece 200. The structure of the appearance inspection device 20 will be described in detail later.

[0068] It should be noted that in Figure 1In this case, an arc welding device for performing arc welding is exemplified as the welding device 10, but it is not particularly limited thereto. For example, the welding device 10 may also be a laser welding device for performing laser welding. In this case, instead of the torch 11, a laser head (not shown) connected to a laser oscillator (not shown) via an optical fiber (not shown) is mounted and held on the robot 16. Thereby, high-output laser for processing is irradiated from the laser head to the workpiece 200, and the workpiece 200 is heated to perform laser welding. In addition, in the following description, the torch 11 and the laser head may be collectively referred to as the welding head 11.

[0069] [Structure of Appearance Inspection Device]

[0070] Figure 3 A functional block diagram of the appearance inspection device is shown. Figure 4 It is a schematic diagram showing a case where the shape of the weld seam is measured by the shape measurement unit. Figure 5A A schematic diagram showing the hardware structure of the sensor control unit. Figure 5B A schematic diagram showing the hardware structure of the data processing unit.

[0071] The shape measurement unit 21 is, for example, a three-dimensional shape measurement sensor composed of a laser light source 21a and a light receiving sensor matrix 21b. The laser light source 21a irradiates the surface of the workpiece 200 with measurement outgoing light, that is, laser, and is configured to be able to scan the surface of the workpiece 200. The light receiving sensor matrix 21b captures the reflection trajectory of the laser irradiated on the surface of the workpiece 200 (hereinafter sometimes referred to as a shape line).

[0072] As Figure 4 As shown, the shape measurement unit 21 scans a specified area including the welding part 201 and its surroundings with laser (outgoing light), and the light receiving sensor matrix 21b captures the reflected light of the outgoing light reflected on the surface of the workpiece 200, thereby measuring the three-dimensional shape of the welding part 201. It should be noted that the welding part 201 is a so-called weld seam formed along the following direction, which is the direction extending along the welding line preset by a welding program or the like. In other words, the weld seam represents welding marks such as bulges and deformations generated in the welded part of the workpiece 200 during welding by welding methods such as arc welding and laser welding. More specifically, it represents the metal part after melting and solidification during welding. In addition, the welding line is an imaginary line when the weld seam or the welded part is represented as a line. In the following description, the direction extending along this welding line is sometimes referred to as the Y direction. On the other hand, the direction orthogonal to the Y direction on the surface of the workpiece 200 where the welding part 201 is formed is sometimes referred to as the X direction. In addition, the height direction of the welding part 201 with respect to the surface of the workpiece 200 is sometimes referred to as the Z direction. The Z direction is orthogonal to the X direction and the Y direction respectively.

[0073] It should be noted that in the description of this application, "orthogonal", "parallel", or "identical" means "orthogonal", "parallel", or "identical" including manufacturing tolerances of components constituting the welding system 100, assembly tolerances, machining tolerances of the workpiece 200, and deviations in the moving speed of the robot 16. It does not mean that the comparison objects are orthogonal, parallel, or equal in the strict sense.

[0074] Taking Figure 4 the example shown, the emitted light emitted from the laser light source 21a is irradiated onto a plurality of points along the width direction of the welding portion 201, in this case along the X direction. The laser irradiated onto the plurality of points is respectively reflected, and the reflected light is photographed by the light receiving sensor matrix 21b. In addition, the shape measurement unit 21 held on the robot 16 moves along the Y direction at a specified speed, and during the movement, emits the emitted light onto the welding portion 201 and its surroundings at specified time intervals, and each time the reflected light is photographed by the light receiving sensor matrix 21b.

[0075] It should be noted that as described above, the shape measurement unit 21 is configured to perform shape measurement not only on the welding portion 201 but also on its surroundings within a specified range. This is to evaluate the presence or absence of spatter 204 and dirt 206 (refer to Figure 6A ).

[0076] Here, the "measurement resolution" refers to the distance between adjacent measurement points in the shape data measured by the shape measurement unit 21. For example, the measurement resolution in the X direction is the distance along the X direction between adjacent measurement points. The measurement resolution in the X direction is set according to the performance of the shape measurement unit 21, mainly the performance of the light receiving sensor matrix 21b, specifically the size of each sensor included in the light receiving sensor matrix 21b in the X direction and the distance between the sensors.

[0077] The measurement resolutions are respectively set in the X direction, Y direction, and Z direction. As described later, the measurement resolution in the Y direction changes according to the moving speed of the robot 16 or according to the sampling frequency of the light receiving sensor matrix 21b.

[0078] In addition, when only referred to as "resolution", the resolution refers to the interval between adjacent coordinate points in the multiple point cloud data of the welded part 201 obtained by the shape measurement unit 21. As will be described later, the shape data is reconstructed according to the shape of the welded part 201. The resolution of the shape data before reconstruction is equal to the above-mentioned measurement resolution. On the other hand, the resolution of the shape data after reconstruction may be different from the measurement resolution. In the example shown in the specification of the present application, the X-direction resolution of the shape data is equal to the X-direction measurement resolution. On the other hand, the Y-direction resolution of the shape data may be different from the Y-direction measurement resolution. Resolutions are set in the X direction, Y direction, and Z direction respectively.

[0079] As Figure 5A shown, the sensor control unit 22 includes at least a CPU 22a and a RAM 22b. In the sensor control unit 22, the CPU 22a sends control commands to the shape measurement unit 21 to control the operation of the shape measurement unit 21. Among the control commands sent from the CPU 22a to the shape measurement unit 21, for example, there are inspection conditions of the shape measurement unit 21, measurement start commands, measurement stop commands, etc. The inspection conditions set in advance are stored in the RAM 22b. It should be noted that other data may also be stored. In addition, the sensor control unit 22 may also include Figure 5A components other than those shown. For example, it may also include a ROM and an HDD as storage devices.

[0080] The data processing unit 23 receives the point cloud data of the shape line obtained by the shape measurement unit 21 as shape data and processes it.

[0081] In addition, as Figure 3 shown, the data processing unit 23 is composed of multiple functional blocks. Specifically, the data processing unit 23 has a shape data processing unit 24, a first storage unit 25, a first determination unit 28, and a notification unit 29.

[0082] In addition, as Figure 5B shown, the data processing unit 23 includes at least a CPU 23a, a GPU (Graphics Processing Unit, graphics processor) 23b, a RAM / ROM 23c, an IC 23d, an input port 23e, an output port 23f, and a data bus 23h as hardware. In addition, the data processing unit 23 includes a display 23g.

[0083] Figure 5B The hardware structure of the data processing unit 23 shown is the same as that of a well-known personal computer (PC). In addition, Figure 3 the multiple functional blocks within the data processing unit 23 shown are through Figure 5BIt is implemented by executing specified software in various devices shown, especially in CPU 23a and GPU 23b. It should be noted that although Figure 5B an example is shown where various devices are connected to a single data bus 23h, multiple data buses can also be provided according to the usage, similar to a normal PC.

[0084] In the data processing unit 23, the shape data processing unit 24 has a function of removing noise from the shape data acquired by the shape measurement unit 21. Since the reflectivity of the emitted light emitted from the shape measurement unit 21 varies depending on the material of the workpiece 200, when the reflectivity is too high, halos or the like are caused to form noise, which may sometimes affect the shape data. Therefore, in the shape data processing unit 24, it is configured to perform noise filtering processing by software. It should be noted that by providing an optical filter (not shown) to the shape measurement unit 21 itself, noise can also be removed. By using the optical filter and the software-based filtering processing together, high-quality shape data can be obtained. In addition, thereby, the quality of the determination model of the learning data set described later can be improved, and the shape of the welding part 201 can be determined with high accuracy as being good or not.

[0085] The noise removal function of the shape data processing unit 24 is mainly implemented by the IC 23d of the data processing unit 23. However, it is not particularly limited thereto. For example, the GPU 23b of the data processing unit 23 can also remove noise from the shape data.

[0086] The shape data processing unit 24 corrects the inclination, deformation, etc. of the base part of the welding part 201 with respect to a specified reference plane, such as the mounting surface of the workpiece 200, by performing statistical processing on the point cloud data. In addition to this, for example, in order to emphasize the shape and position of the welding part 201, edge enhancement correction of the periphery of the welding part 201 is sometimes performed.

[0087] In addition, the shape data processing unit 24 extracts feature amounts of the shape data according to the shape of the workpiece 200 and according to the inspection items of the shape of the welding part 201. In this case, for one shape data, one or more feature amounts corresponding to one or more inspection items are extracted. In addition, the extracted feature amounts are associated with the shape data and are used for subsequent data processing. Here, the feature amount refers to various specific factors extracted from the shape data. As representative feature amounts, there are the length, width, height from the reference plane of the welding part 201, and the differences in length, width, and height between multiple points within the welding part 201. However, it is not particularly limited thereto, and the feature amounts are appropriately set according to the content determined in each inspection item.

[0088] In addition, the shape data processing unit 24 is configured to be able to perform reconstruction of the acquired shape data. Specifically, the shape data processing unit 24 is configured to be able to perform resolution conversion correction on the acquired shape data. Reconstruction of the shape data and resolution transformation correction will be described in detail later.

[0089] The edge enhancement correction processing function, feature amount extraction function, and reconstruction / resolution conversion correction function of the shape data processing unit 24 are mainly implemented by the CPU 23a of the data processing unit 23. However, it is not particularly limited thereto. For example, a part or all of the edge enhancement correction processing may be performed by the IC 23d or the GPU 23b.

[0090] In the first storage unit 25, shape data of the welding part 201 in other workpieces 200 that have been processed before welding the workpiece 200 to be evaluated is stored. In addition, shape data obtained in advance through experiments before actually welding the workpiece 200 may also be stored in the first storage unit 25. In the following description, these shape data obtained in advance are sometimes referred to as sample shape data.

[0091] The sample shape data includes qualified product data in which the shape of the welding part 201 to be evaluated is good and unqualified product data in which there are some defects in the shape. Further, the qualified product data is sample shape data that does not include shape defects in the welding part 201. The unqualified product data is sample shape data that includes shape defects in the welding part 201.

[0092] It should be noted that the shape data of the welding part 201 in other workpieces 200 and the shape data of the welding part 201 in the workpiece 200 to be evaluated are of course obtained for the same welding part 201 in workpieces 200 having the same shape and material.

[0093] In addition, when acquiring the sample shape data, the inspection conditions of the shape measurement unit 21 are fixed. However, the inspection conditions may also be changed according to each material of the workpiece 200 or each shape of the workpiece 200. In general, the shape measurement results of a linear weld are acquired as the sample shape data. That is, regardless of the position in the width direction (X direction) of the weld, the Y-direction resolution of the sample shape data is constant. Therefore, in the learning data described later, the Y-direction resolution is also constant regardless of the position in the width direction (X direction) of the weld.

[0094] In addition, the first storage unit 25 stores a determination model for evaluating the shape of the welding part. The determination model is prepared separately for each material and shape of the workpiece 200. The determination model may also be prepared for each inspection item of the welding part 201. The determination model will be described later.

[0095] The first determination unit 28 determines whether the shape of the welded part 201 is good or not based on the shape data of the welded part 201 that has been processed by the shape data processing unit 24 for noise removal, edge enhancement, etc. and reconstructed as needed, and the determination model corresponding to the selected inspection item. In other words, the first determination unit 28 determines whether the shape data of the welded part 201 obtained by the shape measurement unit 21 meets the specified determination criteria. The determination model is represented by a combination of multiple recognizers with weighted values respectively. For example, the determination model is a well-known object detection algorithm represented by CNN (Convolutional Neural Network).

[0096] The determination of whether the welded part 201 is good or not is generally performed according to the following steps. First, the shape data of the welded part 201 obtained by the shape measurement unit 21 and pre-processed by the data processing unit 23 and the shape data processing unit 24 becomes the input data input to the first determination unit 28.

[0097] In the first determination unit 28, the determination model read from the first storage unit 25 is set in the first determination unit 28. It should be noted that the determination model has been strengthened through learning using a pre-prepared learning data set.

[0098] A learning data set is generated based on the above sample shape data. Specifically, a learning data set is generated using qualified product data and unqualified product data. The unqualified product data is processed into a state marked with various shape defects while changing the number and position of the shape defects, and the processing result is used as multiple learning data. The marked learning data and the qualified product data are summarized and used as the learning data set. It should be noted that, as described later, the learning data set can also be generated inside the data processing unit 23.

[0099] The thus-prepared learning data set is input into the determination model, and the determination result is manually confirmed by a welding operator or the like. In the case where the type of welding defect does not match the learning data, annotation is performed. It should be noted that annotation refers to the process of marking the presence or absence of shape defects and the type of shape defects determined by visually observing the actual welded part 201 on the corresponding part of the shape data. This annotation is also performed manually.

[0100] By performing annotation, the presence or absence of shape defects and the type of shape defects are re-evaluated in the learning data. Based on the execution result, the learning data set is remade or newly made, and further, the determination model is re-learned using the annotated learning data set. By performing these operations one or more times repeatedly, the determination model is strengthened through learning.

[0101] If the shape data of the welding part 201 is input into the determination model set in the first determination unit 28, the determination result of whether there is a shape defect in the shape data of the welding part 201 is output, and in the case of the existence of a shape defect, the type of the shape defect is output as the determination result. However, as will be described later, since there are multiple modes of shape defects, in fact, the type of shape defect included in the shape data is calculated in the form of probability. If the probability is above a specified value, it is determined that there is a shape defect, and the type of the shape defect is determined. This will be described in detail later.

[0102] For example, the consistency of the type of shape defect annotated in the learning data and the type of shape defect included in the shape data of the welding part 201 is determined by probability. On this basis, when the probability exceeds a specified threshold, the type of shape defect included in the shape data of the welding part 201 is determined.

[0103] The following information is output from the first determination unit 28. That is, it outputs whether there is a shape defect, and in the case of the existence of a shape defect, it outputs the type, number, size of the shape defect, and the position of the shape defect in the welding part 201. Furthermore, in the case where the shape defect exceeds the threshold based on a preset determination criterion, it outputs the result of whether the shape of the welding part 201 is good or bad. It should be noted that this threshold varies according to the type and size of the shape defect. For example, for the spatter described later (refer to Figure 6A ), in the case where there are five or more spatters with a diameter of 5 μm or more, it is determined that the shape of the welding part 201 is bad. For the perforation (refer to Figure 6A 、 Figure 6C ), in the case where there is one or more, it is determined that the shape of the welding part 201 is bad. It should be noted that these are just examples and can be appropriately changed according to the above determination criterion and threshold.

[0104] It should be noted that since there are multiple inspection items for the shape of the welding part 201, it is determined whether each inspection item meets the specified determination criterion. Only when all the inspection items to be determined are satisfied, it is finally determined that the shape of the welding part 201 is good.

[0105] The notification unit 29 is configured to notify the determination result in the first determination unit 28 to the output control unit 15, the robot control unit 17, the welding operator, or the system administrator. For example, the display 23g of the data processing unit 23 corresponds to the notification unit 29. The determination result can be displayed on the display 23g or a display unit (not shown) of the welding system 100, or output from a printer (not shown), or can be notified by both of them. In this case, not only the final determination result but also the determination results of each inspection item, such as the presence or absence of a shape defect, are displayed on the display 23g or the display unit (not shown) of the welding system 100. In the case where there is a shape defect, the type, size, number of the shape defect, and the position of the shape defect in the welding part 201 are also displayed. In this way, the welding operator or the system administrator can specifically know what kind of defect has occurred in the welding part 201.

[0106] It should be noted that if only the final determination result is notified, it can also be output as sound from a sound output unit (not shown).

[0107] It should be noted that it is configured that when the final determination result in the first determination unit 28 is affirmative, that is, it is determined that the shape of the welding part 201 is good, the welding system 100 continuously welds the next welding part 201 in the same workpiece 200 or welds the same welding part 201 in the next workpiece 200.

[0108] On the other hand, when the final determination result in the first determination unit 28 is negative, that is, it is determined that the shape of the welding part 201 is defective, the output control unit 15 stops the welding output of the welding torch 11, and the robot control unit 17 stops the operation of the robot 16, or operates the robot 16 in such a way that the welding torch 11 comes to a specified initial position.

[0109] [Regarding the shape defect of the welding part]

[0110] Figure 6A - 6E An example of the shape defect mode of the welding part is shown respectively. It should be noted that Figure 6A - 6E shows the shape of the welding part 201 during butt welding, Figure 6A shows the top view shape, Figure 6B - 6E shows along Figure 6A a cross-sectional view taken along the VIB-VIB line to the VIE-VIE line.

[0111] As Figure 6A - 6EAs shown, in the case of arc welding or laser welding of the workpiece 200, due to poor setting of welding conditions or use of low-quality workpieces 200, various shape defects may occur at the welding part 201. For example, sometimes a part of the welding part 201 burns through (hereinafter, a through hole formed on the workpiece 200 by burning through a part of the welding part 201 from the workpiece 200 is sometimes referred to as a perforation 202) or undercut 203 occurs. It should be noted that the undercut 203 refers to a defective part where the edge part of the weld is in a state of being recessed compared to the surface of the workpiece 200.

[0112] In addition, regarding the variation ranges of the length, width, and height from the reference plane of the welding part 201, they sometimes exceed the allowable ranges ΔL, ΔW, ΔH compared to the respective design values L, W, H. This situation is also regarded as a shape defect.

[0113] In addition, sometimes when a molten droplet (not shown) formed at the tip of the welding wire 12 transfers to the workpiece 200, a part of the molten droplet or fine particles of the molten metal of the workpiece 200 scatter to generate spatter 204, or in the case where the workpiece 200 is a galvanized steel sheet, a part evaporates from the welding part 201 to generate a pit 205, or in the case where the workpiece 200 or the welding wire 12 is an aluminum-based material, dirt 206 is generated near the welding part 201.

[0114] It should be noted that the pit 205 is a part that opens on the surface of the weld, and the dirt 206 is a black carbon black-like attachment generated near the weld. Including the above-mentioned perforation 202, undercut 203, spatter 204, etc., they are respectively one of the modes (types) of shape defects.

[0115] In this way, there are various modes of shape defects in the welding part 201, and it is necessary to set judgment criteria for each mode to conduct inspections. For example, for the perforation 202 and undercut 203, it is necessary not only to judge whether they are good or bad based on their presence or absence, but also to set, for example, the contrast or height difference with the surrounding of the welding part 201 for identifying the perforation 202, etc. to judge whether they are good or bad. In addition, for example, regarding the spatter 204, it is necessary to set its average diameter and use the number of spatters 204 with an average diameter of a specified value or more per unit area to judge whether they are good or bad. Moreover, according to the material of the workpiece 200, the welding part, and the customer's required specifications, etc., the number of inspection items and the judgment criteria for the shape of the welding part 201 are changed or added.

[0116] In addition, the criteria for determining whether there is a shape defect based on the shape data vary depending on the material and shape of the workpiece 200. As described above, since the reflectivity of the laser (emitted light) varies depending on the material of the workpiece 200, for example, the brightness level and contrast of the shape data also change. In addition, even when welding straight portions of the same length, the weld shape of the welded portion 201 sometimes changes depending on the shape of the workpiece 200 under the influence of gravity or the like.

[0117] [Appearance inspection steps for the welded portion]

[0118] Figure 7 Flowchart showing the appearance inspection steps for the welded portion.

[0119] First, the shape of the welded portion 201 is measured using the appearance inspection device 20 (step S1), and the shape data processing unit 24 extracts the shape data of the welded portion 201, that is, the shape data of a predetermined range where the weld is formed (step S2). It should be noted that, in order to confirm the presence of spatter 204 and dirt 206, the extraction range of the shape data is a range slightly larger than the formation range of the weld.

[0120] Furthermore, the shape data processing unit 24 reconstructs the extracted shape data (step S3). The specific steps of the reconstruction will be described in detail later. It should be noted that, in the case where the weld is linear, this step is sometimes omitted.

[0121] The reconstructed shape data is input into the determination model set in the first determination unit 28 (step S4), and the first determination unit 28 determines whether the shape of the welded portion 201 is good or not. When making this determination, as described above, the first determination unit 28 determines whether the shape data contains a shape defect (step S5). In addition, the first determination unit 28 determines the size, number, and position of the shape defect in the welded portion 201 (step S6). Furthermore, the first determination unit 28 determines the type of the shape defect (step S7).

[0122] It should be noted that, in step S7, as described above, considering the shape, size, and position of the shape defect in the welded portion 201, etc., the type of the shape defect is determined. In this case, for example, the probability that the shape defect is spatter 204 is calculated, and if this probability is equal to or greater than a specified value (for example, 70%), it is determined that the shape defect is spatter 204.

[0123] The determination result of the quality of the shape of the final welding part 201 is displayed on the display 23g (step S8). Moreover, not only that, but also the position, size, and number of shape defects in the welding part 201 are displayed on the display 23g. In addition, for the type of shape defect, for example, if it is a spatter 204, it is displayed in red, and if it is a perforation 202, it is displayed in yellow. In addition, when the presence or absence of spatter 204 and the upper limit number of spatter 204 are set as inspection items, the part recognized as spatter 204 can be displayed in a color different from the background, and the probability of being spatter 204 can also be displayed in a distinguishable color. Thus, the welding operator or system administrator can easily identify at a glance the presence or absence of such a shape defect part and the distribution degree of the shape defect part. For example, it can also be that if the probability is 30% or less, it is displayed in green, and if the probability is 70% or more, it is displayed in red. It should be noted that, of course, the division of the probability range and the corresponding color setting in this case can also be arbitrarily set. In addition, when the size of the spatter 204 is also included in the determination criterion of the shape quality, of course, the size of the spatter 204 calculated based on the shape data is compared with the determination criterion to determine the quality of the shape.

[0124] In addition, in step S8, the shape of the welding part 201 is displayed on the display 23g as point cloud data.

[0125] The notification unit 29 notifies the output control unit 15 and the robot control unit 17 of the determination result of the quality of the shape of the welding part 201 (step S9). In addition, the notification unit 29 can also notify the welding operator or system administrator of the determination result of the quality of the shape of the welding part 201. As described above, according to the determination result of the first determination unit 28, the welding output of the welding torch 11 and the movement of the robot 16 are controlled.

[0126] [Shape data reconstruction step]

[0127] Figure 8A is a schematic diagram showing the measurement point positions of the shape data before reconstruction, Figure 8B is a schematic diagram showing the measurement point positions of the shape data after reconstruction.

[0128] The shape of the welding part 201 varies according to the shape of the workpiece 200 and the like, and is not necessarily limited to a linear shape. For example, as Figure 8A , Figure 8B shown, when the welding part 201 includes an arc-shaped curved portion, the scanning trajectory of the shape measurement unit 21 along the welding line is of course also a curve. On the other hand, since the time interval of shape measurement is constant, as Figure 8AAs shown, the moving distance of the shape measurement unit 21, in other words, the interval of the measurement points along the Y direction is different between the inner peripheral side and the outer peripheral side of the weld.

[0129] That is, when the welding part 201 is curved, the interval of the measurement points along the Y direction, that is, the Y-direction resolution, becomes larger on the outer peripheral side and smaller on the inner peripheral side.

[0130] In this way, if the Y-direction resolution of the welding part 201 varies according to the position of the welding part 201, there will be a part that is inconsistent with the Y-direction resolution of the learning data. As a result, the accuracy of determining the quality of the shape of the welding part 201 based on the determination model may decrease.

[0131] Therefore, in the present embodiment, the shape data is reconstructed according to the steps shown below. As a result, as Figure 8B shown, even when the welding part 201 includes a curved portion, the Y-direction resolution of the shape data can be made constant, thereby improving the consistency with the Y-direction resolution of the learning data. Thereby, the accuracy of determining the quality of the shape of the welding part 201 based on the determination model can be improved.

[0132] Figure 9 is a flowchart showing the reconstruction steps of the shape data, Figure 9 corresponding to the processing in step S3 shown in Figure 7 shown.

[0133] First, the X-direction resolution of the shape data extracted in step S2 shown in Figure 7 is stored in the first storage unit 25 (step S10). This X-direction resolution is equal to the X-direction measurement resolution of the light-receiving sensor matrix 21b.

[0134] Next, for each measurement point, the position Xi (i is an integer, 1 ≤ i ≤ N; N is the number of sensors) of each sensor included in the light-receiving sensor matrix 21b in the X direction is calculated, and the calculation result is stored in the first storage unit 25 (step S11).

[0135] In the light-receiving sensor matrix 21b, the position Xc of the sensor virtually arranged at the center is calculated for each measurement point, and the calculation result is stored in the first storage unit 25 (step S12).

[0136] The shape data processing unit 24 determines whether Xi = Xc based on the values stored in the first storage unit 25 (step S13).

[0137] When the determination in step S13 is affirmative, that is, Xi = Xc, it proceeds to step S14, where the Y-direction resolution at the position Xc is calculated and stored in the first storage unit 25.

[0138] On the other hand, when the determination in step S13 is negative, the process proceeds to step S15, and the shape data processing unit 24 determines whether Xi>Xc.

[0139] If the determination in step S15 is affirmative, that is, if Xi>Xc, the process proceeds to step S16 , where the Y-direction resolution at the position Xi (i>c) is calculated and stored in the first storage unit 25 .

[0140] If the judgment in step S15 is negative, that is, if Xi<Xc, the process proceeds to step S17 to calculate the position Xi (i <c)处的Y方向分辨率,并保存在第一存储部25中。

[0141] It should be noted that, when the welding part 201 includes a curved portion, the Y-direction resolution Pi varies depending on the position Xi of the sensor in the X-direction. <c的情况下,P i <P c , in the case of i = c, P i =P c , in the case of i>c, P i >P c .

[0142] Y-direction resolution P i The reason why the resolution P varies depending on the position Xi of the sensor in the X direction is that the measurement resolution P in the Y direction of the light receiving sensor matrix 21 b varies depending on the position Xi of the sensor in the X direction. This point will be described further.

[0143] Figure 10 This is a schematic diagram showing the relationship between the Y-direction resolution and the circumferential speed at the curved portion of the weld, and the distance from the center of the curved portion to the measurement point in the X direction.

[0144] like Figure 10 As shown, the center O is virtually set for the arc-shaped curved portion of the welded portion 201. The center position X in the X direction of the welded portion 201 is c The trajectory corresponds to the movement trajectory of the front end of the welding torch 11 achieved by the robot 16.

[0145] like Figure 10 As shown, by arranging at the center position X c The distance l between adjacent measurement points measured by the sensor c (hereinafter, sometimes referred to as the measurement point interval l in the Y direction c . ) is expressed by mathematical formula (1).

[0146] [Mathematical formula 1]

[0147] l c = r c θ ···(1)

[0148] Here, θ is the angle between the points adjacent to each other among the measurement points of the sensor arranged at the central position X0 and the center O (hereinafter, sometimes referred to as the central angle θ0). In addition, r c is the distance along the X direction from the center O to the movement locus of the sensor arranged at the central position X c . That is, when the center of the arc-shaped curved portion of the welding portion 201 is set to O, r c is the radius of the movement locus of the sensor arranged at the central position X c in the X direction.

[0149] On the other hand, based on the movement locus of the sensor at the central position Xc, the measurement point interval l near in the Y direction of the sensor arranged on the side closer to the center O is represented by the mathematical formula (2), and the measurement point interval l far in the Y direction of the sensor arranged on the side farther from the center O is represented by the mathematical formula (3).

[0150] [Mathematical formula 2]

[0151]

[0152] [Mathematical formula 3]

[0153]

[0154] Here, ΔR is the distance along the X direction from the movement locus of the sensor at the position X1 closest to the center O to the movement locus of the sensor at the position X N farthest from the center O (refer to Figure 11 ).

[0155] Here, if the time for the light-receiving sensor matrix 21b of the shape measurement unit 21 to move between the measurement points is t (seconds), then the moving speed v (mm / s) along the Y direction on each movement locus (hereinafter, sometimes referred to as the circumferential speed v0) is represented by the mathematical formulas (4) to (6).

[0156] [Mathematical formula 4]

[0157]

[0158] [Mathematical formula 5]

[0159]

[0160] [Mathematical formula 6]

[0161]

[0162] As can be seen from mathematical formulas (4) to (6), the circumferential velocity v depends on the radius r from the center 0 to the movement trajectory of the sensor (hereinafter, sometimes simply referred to as the radius r).

[0163] Here, the radius and circumferential velocity on the movement trajectory of the sensor located at the center position Xc are respectively set as r c , v c . The radius and circumferential velocity on the movement trajectory closest to the center 0 are respectively set as r near , V near . The radius and circumferential velocity on the movement trajectory farthest from the center O are respectively set as r far , V far .

[0164] As can be seen from mathematical formulas (4) to (6), regarding the circumferential velocity v, the relationship shown in mathematical formula (7) holds.

[0165] [Mathematical formula 7]

[0166] v near ≤v≤v far …(7)

[0167] Here, if the sampling frequency during shape measurement is set as f s (Hz), then the measurement resolution P in the Y direction is expressed by mathematical formula (8).

[0168] [Mathematical formula 8]

[0169]

[0170] In addition, the measurement resolution of the sensor arranged at the center position X c is set as P c , the measurement resolution of the sensor arranged on the side closest to the center O is set as P near , and the measurement resolution of the sensor arranged on the side farthest from the center 0 is set as P far . The ratio of P c to P near , and the ratio of P c to P far are respectively expressed by mathematical formulas (9) and (10).

[0171] [Mathematical formula 9]

[0172]

[0173] [Mathematical formula 10]

[0174]

[0175] In addition, P c and P near The relationship between and P c and P far The relationships between are represented by mathematical expressions (11) and (12) respectively.

[0176] [Mathematical Expression 11]

[0177]

[0178] [Mathematical Expression 12]

[0179]

[0180] From mathematical expressions (11) and (12), it can be seen that for the measurement resolution P in the Y direction, the relationship shown in mathematical expression (13) holds.

[0181] [Mathematical Expression 13]

[0182] P near ≤ P ≤ P far ···(13)

[0183] From mathematical expressions (11) to (13), it can be seen that the measurement resolution P in the Y direction also varies according to the above radius r. In addition, the Y-direction resolution P i corresponds to the measurement resolution P in the Y direction. Therefore, the Y-direction resolution P i also varies according to the radius r.

[0184] Summarize the above situations in Figure 11 for illustration. Figure 11 is a schematic diagram showing the relationship between the measurement point positions at the curved part of the welding area and various parameters.

[0185] Next, the Y-direction resolution P calculated in steps S14, S16, and S17 respectively will be specifically described. i Specifically explain.

[0186] In the process of step S17, the subscript i is less than c. In other words, it is the case where the distance between the position Xi and the center O is less than the distance between the center position Xc of the sensor array and the center O. In this case, the Y-direction resolution P i is represented by mathematical expression (14).

[0187] [Mathematical Expression 14]

[0188]

[0189] ΔR is the distance from the movement locus of the sensor at the position X1, which is arranged on the first side, i.e., the side closest to the center O, to the movement locus of the sensor at the position XN, which is arranged on the Nth side, i.e., the side farthest from the center O. Δs is the interval in the X direction between the movement loci of two adjacent sensors in the X direction. P c is the Y-direction resolution at the center position Xc, and rc is the distance in the X direction from the center O to the movement locus of the sensor arranged at the position Xc.

[0190] In the process of step S14, the subscript i is the same as c. In other words, it is the case where the position Xi is at the same position as the center position Xc of the sensor array. In this case, the Y-direction resolution P i is represented by the mathematical formula (15).

[0191] [Mathematical formula 15]

[0192]

[0193] In this case, the Y-direction resolution P i (=P c ) is equal to the value obtained by dividing the circumferential speed v c of the sensor at the center position Xc by the sampling frequency f s .

[0194] In the process of step S16, the subscript i is greater than c. In other words, it is the case where the distance between the position Xi and the center O is greater than the distance between the center position Xc of the sensor array and the center O. In this case, the Y-direction resolution P i is represented by the mathematical formula (16).

[0195] [Mathematical formula 16]

[0196]

[0197] It should be noted that the expression of the Y-direction resolution P shown in the mathematical formulas (14) to (16) i can also be applied to the case where the welding part 201 is linear. In this case, since r c can be regarded as infinity, in the mathematical formulas (14) and (16), the terms containing r c all approach zero, and any of the mathematical formulas (14) to (16) is expressed as the mathematical formula (15), that is, the Y-direction resolution P i =P c .

[0198] When the determination result in step S18 is negative, the process returns to step S13, and the series of processes of steps S13 to S17 are repeatedly executed until the determination result in step S18 becomes positive.

[0199] The shape data processing unit 24 sequentially executes the processes of steps S13 to S17 at each point of the shape data, and determines whether the arithmetic process of calculating and storing the Y-direction resolution has been executed at all measurement positions in the welding part 201 (step S18).

[0200] When the determination result in step S18 is positive, the shape data processing unit 24 performs resolution correction of the shape data based on the arithmetic results of steps S12, S15, S16, and S17 (step S19).

[0201] Specifically, focusing on the Y-direction measurement resolution of the shape measurement unit 21, the Y-direction resolution of the shape data is corrected so that the Y-direction resolution of the measured shape data reaches a value equal to the Y-direction resolution of the learning data used for learning reinforcement of the determination model, and further equal to the Y-direction resolution of the sample shape data previously obtained for generating the learning data. In this case, the coordinate position in the Z direction is also corrected at the same time. Thus, by correcting the resolution and reconstructing the shape data, even when the welding part 201 includes a curved portion, the first determination unit 28 can accurately determine whether the shape of the welding part 201 is good or not. Hereinafter, this point will be further described.

[0202] Figure 12A Shows the three-dimensional coordinate positions of the measurement points before and after reconstructing the shape data, Figure 12B Shows viewing from the Z direction Figure 12A Time graph. Figure 13 It is a conceptual diagram showing the derivation steps of the coordinate points of the shape data based on bilinear interpolation.

[0203] When reconstructing the shape data, as Figure 12A , Figure 12B As shown, the position of the reconstructed measurement point is derived based on the positions of the actual measurement points adjacent to its surroundings.

[0204] Specifically, the position of the correction coordinate point in the Y direction is corrected so that the Y-direction resolution becomes the above-mentioned P c . Further, by correcting the coordinate position in the Z direction based on the coordinate position in the X direction of each coordinate point and the corrected coordinate position in the Y direction, the shape data is reconstructed. Hereinafter, the reconstruction step of the coordinate position of the measurement point in the Z direction will be described.

[0205] First, as shown in mathematical expressions (17) and (18), interpolation parameters t and s are respectively derived.

[0206] [Mathematical formula 17]

[0207]

[0208] [Mathematical formula 18]

[0209]

[0210] Here, x1 and x2 are the X - coordinate positions of the surrounding measurement points before reconstruction, and y1 and y2 are the Y - coordinate positions of the surrounding measurement points before reconstruction. P xmeas and P ymeas are the X - direction resolution and Y - direction resolution of the shape data respectively. P xai and P yai are the X - direction resolution and Y - direction resolution of the shape data after reconstruction respectively.

[0211] Next, using the interpolation parameters t and s, the Z - coordinate positions of the measurement points after reconstruction are corrected. In this embodiment, bilinear interpolation is used for this correction. The Z - coordinate positions of the measurement points after correction are represented by Mathematical formula (19). It should be noted that the Z - coordinate positions of the surrounding measurement points before reconstruction are represented by f(x1, y1), f(x1, y2), f(x2, y1), and f(x2, y2) respectively.

[0212] [Mathematical formula 19]

[0213]

[0214] When the resolution correction of the shape data is completed as described above, the process proceeds to Figure 7 Step S4 shown, and subsequent processing is executed.

[0215] It should be noted that in this embodiment, the resolution correction of the shape data using bilinear interpolation is described, but other methods can also be used. For example, spline interpolation or bicubic interpolation can be used.

[0216] [Effects, etc.]

[0217] As described above, the appearance inspection device 20 according to this embodiment inspects the appearance of the welded part 201 of the workpiece 200.

[0218] The appearance inspection device 20 includes at least a shape measurement unit 21 and a data processing unit 23. The shape measurement unit 21 is mounted on the robot 16 and measures the three - dimensional shape of the welded part 201 along the welding line, and the data processing unit 23 processes the shape data obtained by the shape measurement unit 21.

[0219] The data processing unit 23 further includes a shape data processing unit 24 and a first determination unit 28. The shape data processing unit 24 reconstructs the shape data, and the first determination unit 28 determines whether the shape of the welding part 201 is good or not based on the shape data reconstructed by the shape data processing unit 24 and one or more determination models prepared in advance.

[0220] The shape measurement unit 21 performs multi-point measurement of the shape of the welding part 201 along the X direction and also performs multi-point measurement of the shape of the welding part 201 along the Y direction, which is the direction along the welding line.

[0221] When the welding part 201 includes a curved portion, the shape data processing unit 24 corrects the coordinate points of the shape data located at different positions along the X direction so that the Y-direction resolution of the shape data is the same. Further, the shape data processing unit 24 corrects the coordinate position in the Z direction of the shape data based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, thereby reconstructing the shape data.

[0222] By configuring the appearance inspection device 20 in this way, when the welding part 201 includes a curved portion, there is no need to separately prepare a determination model for evaluating the shape of the welding part 201. That is, even if only sample shape data and a determination model are used, the three-dimensional shape of the welding part 201 can be evaluated with good accuracy. The above sample shape data includes simple shapes, such as a linear weld seam, and the determination model is a determination model strengthened through learning using learning data obtained based on the sample shape data. Thus, even when the welding part 201 includes a curved portion, it is possible to accurately determine whether the shape of the welding part 201 is good or not.

[0223] In addition, according to the present embodiment, it is possible to match the shape defects included in the sample shape data, and thus the learning data and the reconstructed shape data, such as perforations 202, spatter 204, etc. Therefore, when the reconstructed shape data is input into the learned determination model, it is possible to reliably and accurately determine whether the shape of the welding part 201 is good or not.

[0224] When the welding part 201 has a specified width in the X direction, the shape data processing unit 24 corrects the Y-direction resolution of the shape data so that the Y-direction resolution of the shape data is equal to the Y-direction resolution on the trajectory of the center position X of the welding part 201 in the X direction. c of the trajectory.

[0225] In this way, it is possible to simply correct the Y-direction resolution of the shape data and reduce the influence of deformation of the shape data caused by the correction. As a result, it is possible to accurately evaluate the three-dimensional shape of the welded part 201 and accurately determine whether the shape of the welded part 201 is good or not. Furthermore, it is possible to simply correct the coordinate position in the Z direction of the shape data.

[0226] When determining whether the shape of the welded part 201 is good or not, the first determination unit 28 determines whether there is a shape defect in the input shape data. When making this determination, a learning data set is generated using sample shape data, which consists of qualified product data without shape defects and unqualified product data with a certain shape defect. In the learning data set, the unqualified product data is processed as follows. In this processing, the type of the shape defect is determined and the determined type is marked for the shape defect. Using this learning data set, the determination model is strengthened in advance through learning.

[0227] In addition, when there is a shape defect in the shape data, the first determination unit 28 determines the number, size, and position of the shape defect in a specified area around the welded part 201 and the welded part 201 itself.

[0228] Furthermore, the first determination unit 28 determines the type of the shape defect. When making this determination, the number, size, and / or position of the shape defect in the welded part 201 are referred to. In addition, the type of the shape defect is calculated by probability, and when the probability reaches or exceeds a specified threshold value, the type of the shape defect is determined. It should be noted that the type of the shape defect is not limited to Figure 6A - 6E the defects shown. The case where the size of the welded part 201 does not meet the specified qualified product standard is also included in the shape defect. The qualified product standard for this size can be set in any one of the X direction, Y direction, and Z direction.

[0229] As described above, for the shape of the welded part 201, the first determination unit 28 respectively determines or determines multiple items, and based on these results, finally determines whether the shape of the welded part 201 is good or not. In this way, it is possible to accurately evaluate whether the shape of the welded part 201 is good or not.

[0230] It should be noted that in the present embodiment, the determination model for determining the presence or absence of a shape defect and the determination model for determining the type of the shape defect, etc. are formed as one. However, the above two determination models can also be formed as two.

[0231] The welding system 100 according to the present embodiment includes a welding device 10 for welding a workpiece 200 and an appearance inspection device 20.

[0232] By configuring the welding system 100 in this way, it is possible to accurately inspect the shape of the welded part 201 with less man-hours. As a result, the cost of the welding process can be reduced.

[0233] In addition, the welding apparatus 10 at least includes a welding head 11 (torch 11), a robot 16, an output control unit 15, and a robot control unit 17. The welding head 11 (torch 11) is configured to input heat to the workpiece 200. The robot 16 holds the welding head 11 (torch 11) and moves the welding head 11 (torch 11) to a desired position. The output control unit 15 controls the welding output of the welding head 11 (torch 11). The robot control unit 17 controls the operation of the robot 16.

[0234] When the first determination unit 28 of the appearance inspection apparatus 20 determines that the shape of the welded part 201 is defective, the output control unit 15 stops the welding output of the welding head 11 (torch 11), and the robot control unit 17 stops the operation of the robot 16, or operates the robot 16 so that the welding head 11 (torch 11) reaches a specified initial position.

[0235] By configuring the welding system 100 in this way, when the shape of the welded part 201 is defective, it is possible to stop the subsequent welding and prevent the frequent occurrence of defective products. It should be noted that by obtaining the determination results in the first determination unit 28 for each inspection item, it is possible to infer the defective part in the welding system 100, quickly eliminate the cause of the defect, and shorten the downtime of the welding system 100.

[0236] The appearance inspection method for the welded part according to the present embodiment at least includes: a first step ( Figure 7 step S1), in this first step, the shape measurement unit 21 measures the three-dimensional shape of the welded part 201 while moving together with the robot 16, and obtains shape data; a second step ( Figure 7 step S3), in this second step, the shape data processing unit 24 reconstructs the shape data; and a third step ( Figure 7 steps S5 to S7), in this third step, the first determination unit 28 determines whether the shape of the welded part 201 is good or bad based on the shape data reconstructed by the shape data processing unit 24 and one or more determination models prepared in advance.

[0237] In the second step, when the welding part 201 includes a curved portion, the shape data processing unit 24 corrects the coordinate points of the shape data located at different positions along the X direction so that the Y-direction resolution is the same. Further, the shape data processing unit 24 corrects the coordinate position in the Z direction of the shape data based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, thereby reconstructing the shape data.

[0238] According to the present embodiment, even when the welding part 201 includes a curved portion, the three-dimensional shape of the welding part 201 can be evaluated with good accuracy, and it is possible to accurately determine whether the shape of the welding part 201 is good or bad. In addition, it is possible to match the characteristics of the shape of the defective part included in the learning data and the reconstructed shape data. Therefore, when the reconstructed shape data is input into the learned determination model, it is possible to reliably and accurately determine whether the shape of the welding part 201 is good or bad.

[0239] In the second step, when the welding part 201 has a predetermined width in the X direction, the shape data processing unit 24 corrects the Y-direction resolution of the shape data so that the Y-direction resolution of the shape data is the same as the Y-direction resolution on the trajectory of the center position X of the X direction of the welding part 201 c is the same.

[0240] In this way, it is possible to simply correct the Y-direction resolution of the shape data and reduce the influence of the deformation of the shape data caused by the correction. Thereby, the three-dimensional shape of the welding part 201 can be evaluated with good accuracy, and it is possible to accurately determine whether the shape of the welding part 201 is good or bad. Further, it is possible to simply correct the coordinate position in the Z direction of the shape data.

[0241] The third step includes: a first determination unit 28 determining whether there is a defective shape in the input shape data; when there is a defective shape in the shape data, determining the number, size, and position of the defective shape in a predetermined area around the welding part 201 and its surroundings; and determining the type of the defective shape.

[0242] When determining whether there is a defective shape, the determination model is strengthened by pre-learning using a separately prepared learning data set. A learning data set is generated based on the sample shape data. The sample shape data includes non-defective product data that does not include a defective shape and defective product data that includes a certain defective shape. In addition, when the defective product data is used as learning data, the type of the defective shape is determined, and the type of the defective shape is marked. In addition, when determining the type of the defective shape, the number, size, and / or position of the defective shape in the welding part 201 are referred to.

[0243] In this way, for the shape of the welded part 201, by separately determining or ascertaining a plurality of items, and based on these determination results and ascertainment results, it is finally determined whether the shape of the welded part 201 is good or not, whereby it is possible to accurately evaluate whether the shape of the welded part 201 is good or not.

[0244] Example 1

[0245] Figure 14A A perspective view of the workpiece involved in Example 1 is shown. Figure 14B Shown as viewed from the Z direction Figure 14A at that time. Figure 15A It is a schematic view showing the measurement point positions of the shape data before reconstruction. Figure 15B It is a schematic view showing the measurement point positions of the shape data after reconstruction. It should be noted that, for the sake of convenience in explanation, in Figure 14A each of the drawings shown hereinafter, the same reference numerals are assigned to the same parts as those in the first embodiment, and detailed descriptions thereof are omitted.

[0246] In the present embodiment, the workpiece 200 is composed of a cover 210 and a roller 220. The roller 220 is a cylindrical member having a rotation shaft 221 at its axis. The cover 210 is a cylindrical member. The roller 220 is inserted into the cover 210 such that the inner surface of the cover 210 is in contact with the side surface (outer circumferential surface) of the roller 220. The outer circumferential end of the roller 220 and the inner circumferential end of the cover 210 are welded, thereby forming a welded part (weld seam) 201 on the upper and lower surfaces of the roller 220.

[0247] Therefore, as Figure 14B shown, the shape of the welded part 201 is circular, and the welded part 201 includes a curved portion. The scanning direction of the shape measurement unit 21, that is, the Y direction, extends along the circular welded part 201. In addition, the X direction is the radial direction of the cover 210 and the roller 220.

[0248] In this case, as Figure 15A shown, with respect to the shape data obtained by the shape measurement unit 21, along the X direction, on the side closer to the axis (center O) of the roller 220, the Y direction resolution becomes smaller, and on the other hand, on the side farther from the axis of the roller 220, the Y direction resolution becomes larger. Therefore, in the case where the shape evaluation is performed using a determination model that has been learned and enhanced based on a linear weld seam, it is not possible to evaluate the shape of the welded part 201 with good accuracy. As a result, it is not possible to accurately determine whether the shape of the welded part 201 is good or not.

[0249] Therefore, in the present embodiment, the shape data was reconstructed by the same method as the method shown in the first embodiment. That is, as Figure 15BAs shown, regardless of the position of the original measurement points in the X direction, the shape data was corrected in such a way that the Y-direction resolution of the shape data was the same. In addition, the coordinate positions of the original measurement points in the X direction were also corrected along with this correction. Further, as described above, the coordinate positions of the measurement points in the Z direction were also corrected, which is not shown in the figure.

[0250] By reconstructing the shape data in this way, it is possible to accurately determine whether the shape of the welded part 201 is good or not based on the reconstructed shape data and the above-described determination model.

[0251] Example 2

[0252] Figure 16 is a perspective view showing the workpiece according to Example 2. Figure 17 is a schematic view showing the measurement steps of the welded part in the conventional method, Figure 18 is a schematic view showing the measurement steps of the welded part in Example 2.

[0253] In the present embodiment, the workpiece 200 is composed of a member 230 and a plate 240. The member 230 is a cylindrical member with a radius of r from the center 0 to the outer circumference. The member 230 is placed on the upper surface of the plate 240 in such a way that the bottom surface of the member 230 is in contact with the upper surface of the plate 240. The outer circumference of the member 230 and the upper surface of the plate 240 are welded, thereby forming a welded part (weld seam) 201 on the contact surface between the member 230 and the plate 240.

[0254] Therefore, as Figure 17 、 Figure 18 shown, the shape of the welded part 201 is circular. The scanning direction of the shape measurement unit 21 in the present embodiment, that is, the Y direction, extends along the circular welded part 201 as Figure 18 shown. In other words, the Y direction extends along the welding line direction.

[0255] When measuring the shape of the welded part 201 shown in Figure 16 by the conventional method, in order to match the resolution of the shape data with the resolution of the learning data for learning reinforcement of the determination model, as Figure 17As shown, measurement is performed multiple times. That is, the shape measurement unit 21 scans in the tangential direction of the welding part 201 and measures the shape of the welding part 201. The shape measurement unit 21 scans again in a direction that is 90 degrees different from the first scanning direction and measures the shape of the welding part 201. The same process is repeated twice, and the shape of the welding part 201 is measured a total of four times. The movement trajectories LP1 to LP4 of the shape measurement unit 21 correspond to the respective sides of the quadrilateral surrounding the welding part 201. Therefore, in the case of measurement by the conventional method, the movement distance required for the shape measurement unit 21 to measure the shape of the welding part 201 is 2r × 4 = 8r.

[0256] On the other hand, in the present embodiment, the shape data is reconstructed by the same method as the method shown in the first embodiment. Therefore, when measuring the shape of the welding part 201, it is only necessary to move the shape measurement unit 21 along the center line in the width direction of the welding part 201 corresponding to the welding line, and its movement trajectory CPl becomes a circle. In this case, the movement distance required for the shape measurement unit 21 to measure the shape of the welding part 201 is 2πr.

[0257] Therefore, compared with the conventional method, the movement distance of the shape measurement unit 21, that is, the inspection track becomes shorter. In addition, the movement speed of the front end of the robot 16 equipped with the shape measurement unit 21 is usually constant. Therefore, the ratio of the inspection track between the present embodiment and the conventional method and the ratio of the time required for inspection (production cycle time) are represented by the mathematical formula (20).

[0258] [Mathematical formula 20]

[0259]

[0260] In addition, it can be seen from the mathematical formula (20) that the production cycle time reduction rate between the present embodiment and the conventional method is represented by the mathematical formula (21).

[0261] [Mathematical formula 21]

[0262] Production cycle time reduction rate = (1 - 0.785) × 100 = 21.5 [%] …(21)

[0263] As described above, in the present embodiment, by adopting the same method as the method shown in the first embodiment, thereby, compared with the conventional method, the production cycle time required for inspection can be reduced by 21.5%.

[0264] It should be noted that from the Figure 17As is known, at the switching portions of the respective movement trajectories, the distance between the movement trajectory and the welding portion 201 becomes farther. Therefore, it may not be possible to reliably measure the shape of the welding portion 201. In this case, for example, it is conceivable that at the switching portion between the movement trajectories LP1 and LP2, after tilting 45 degrees from LP1, the shape measurement unit 21 performs scanning. In this case, the same applies to the switching portions of other movement trajectories, and the shape measurement unit 21 performs scanning after tilting 45 degrees from the immediately preceding movement trajectory.

[0265] However, in this case, since it takes time to switch the movement trajectories, the production cycle of the entire inspection becomes longer.

[0266] On the other hand, by measuring the shape of the welding portion 201 by the same method as the method shown in the first embodiment, it is possible to avoid such an increase in the production cycle.

[0267] Embodiment 3

[0268] Figure 19 is a perspective view of the workpiece according to Embodiment 3.

[0269] Figure 19 The workpiece 200 shown is a hemispherical member, and a welding portion 201 is formed on its surface.

[0270] The same method as the method shown in the first embodiment can also be applied to evaluate the shape of the welding portion 201 formed on the workpiece 200 having the shape shown in Figure 19 shown.

[0271] That is, by reconstructing the shape data in the same manner as shown in the first embodiment, it is possible to accurately determine whether the shape of the welding portion 201 is good or bad based on the determination model, where the determination model is a determination model strengthened by learning using the reconstructed shape data and learning data generated from a linear weld seam.

[0272] (Second Embodiment)

[0273] Figure 20 FIG. 21 shows a functional block diagram of the appearance inspection apparatus according to the second embodiment, and FIG. 22 shows a flowchart of the appearance inspection steps of the welding portion.

[0274] Figure 20 The appearance inspection apparatus 20 of the present embodiment shown in Figure 3 is different from the appearance inspection apparatus 20 of the first embodiment shown in

[0275] The learning dataset generation unit 26 reads out the sample shape data stored in the first storage unit 25 and classifies it according to each material and shape of the workpiece 200. Additionally, it can also be classified according to each inspection item of the welding part 201. In this case, the same shape data can be included in different inspection items respectively. Further, the learning dataset generation unit 26 generates a learning dataset, that is, a group of learning data, according to each material and shape of the workpiece 200 based on the feature quantities associated with the sample shape data. For example, the materials and shapes of the workpiece 200 are arranged in the form of a matrix to determine the classification categories, and the learning dataset is classified corresponding to these categories (refer to Figure 20 ). It should be noted that examples of the shape of the workpiece 200 include butt joints, overlap joints, T-joints, cross joints, etc. of plates.

[0276] In addition, the learning dataset generation unit 26 performs data augmentation processing on the sample shape data read out from the first storage unit 25 to generate a learning dataset. Specifically, data augmentation processing is performed by changing one or more feature quantities associated with the sample shape data, or changing the position, number, size of the shape defects in the sample shape data of the welding part 201, or performing both. The generation steps of the learning dataset will be described in detail later.

[0277] The function of the learning dataset generation unit 26 is mainly implemented by the CPU 23a of the data processing unit 23. However, it is not particularly limited to this. For example, a part of this function can also be implemented by the GPU 23b.

[0278] The determination model generation unit 27 generates a determination model for each inspection item of the welding part 201 set according to each material and shape of the workpiece 200 based on the determination criteria set in each item. For example, these determination models are the above-mentioned well-known algorithms for object detection.

[0279] In addition, the determination model generation unit 27 repeatedly learns by inputting the learning dataset corresponding to each material and shape of the workpiece 200 in multiple learning datasets into each determination model generated according to each material and shape of the workpiece 200, thereby improving the determination accuracy of each determination model. In this case, for example, the determination model is generated according to the classification categories shown in Figure 20 . It should be noted that learning is repeated until the accuracy rate, recall rate, and precision of the determination model meet the pre-set values.

[0280] In addition, when generating the determination model, the qualified product data and unqualified product data in the sample shape data are appropriately selected and used according to the material and shape of the workpiece 200. Thereby, the generation time of the determination model can be shortened, and the high-precision of the determination model can be achieved. Similarly, when generating the determination model for each inspection item of the welding part 201, the qualified product data and unqualified product data in the sample shape data are appropriately selected and used according to the inspection item. Thereby, the generation time of the determination model can be shortened, and the high-precision of the determination model can be achieved.

[0281] In addition, in Figure 21 the appearance inspection step of the welding part in the present embodiment shown, after step S22 is the same as Figure 7 the appearance inspection step of the welding part in the first embodiment shown. That is, Figure 21 the processes of steps S22 to S30 shown are respectively the same as Figure 7 the processes of steps S1 to S9 shown. Therefore, here, the processes of steps S20 and S21 will be described. It should be noted that steps S20 and S21 only need to be completed before executing step S25, and do not need to be continuously executed with the processes after step S22.

[0282] The learning data set generation unit 26 reads out the sample shape data previously acquired and stored in the first storage unit 25, and generates a learning data set according to each material and shape of the workpiece 200 (step S20). At this time, as described above, data augmentation processing is performed on the sample shape data. Figure 22A to 22C An example of generating a plurality of data with changed feature amounts is shown.

[0283] Figure 22A An example of the production steps of the learning data set is shown, Figure 22B Another example of the production steps of the learning data set is shown, Figure 22C Still another example of the production steps of the learning data set is shown.

[0284] For example, as Figure 22A shown, a plurality of data obtained by changing the length and position of the welding part 201, which is one of the feature amounts, in the original sample shape data are generated as the learning data set. It should be noted that in Figure 22A , an example of generating a plurality of learning data whose shortening amplitude exceeds the allowable range ΔL compared with the reference value L of the length of the welding part 201 is shown, but it is not particularly limited thereto, and learning data whose lengthening amplitude exceeds the allowable range ΔL compared with the reference value L of the length can also be generated separately.

[0285] Or, as Figure 22BAs shown, a plurality of data obtained by changing the dimensions and positions of the perforations 202 in the original sample shape data are generated as a learning data set. In this case, as feature quantities, the height from the reference plane and the difference in this height between a plurality of points within the welded portion 201 are extracted and varied. Further, as Figure 22C shown, by extracting the same feature quantities around the welded portion 201 and generating a learning data set based on this feature quantity, it is possible to determine whether the spatter 204 and the dirt 206 are present beyond the allowable range.

[0286] Next, the determination model generation unit 27 generates a determination model for each material and shape of the workpiece 200 (step S21). At this time, each generated determination model is strengthened by learning based on the learning data set generated in step S20 and the shape data annotated with the types of shape defects and the like. The determination model after the learning strengthening is used for the processing after step S25.

[0287] It should be noted that, in the present embodiment, the reconstruction of the shape data corresponding to the shape of the welded portion 201 and the resolution correction are also performed, which is the same as in the first embodiment.

[0288] Therefore, there is no need to separately generate a learning data set or generate a determination model only because the shape of the welded portion 201 includes a curved portion.

[0289] In view of this, the learning data set generation unit 26 may at least classify the plurality of sample shape data obtained by the shape measurement unit 21 for each material of the workpiece 200, and perform data augmentation processing on the classified sample shape data to generate a plurality of learning data sets.

[0290] In addition, the determination model generation unit 27 may use a plurality of learning data sets to generate a determination model at least for each material of the workpiece 200.

[0291] According to the present embodiment, the same effects as those achieved by the structure shown in the first embodiment can be achieved. That is, even by using only the sample shape data and the determination model, it is possible to accurately evaluate the three-dimensional shape of the welded portion 201, the above sample shape data includes a linear weld seam, and the determination model is a determination model strengthened by learning using a learning data set obtained based on the sample shape data. That is, even when the welded portion 201 includes a curved portion, it is possible to accurately determine whether the shape of the welded portion 201 is good or bad.

[0292] In addition, it is possible to match the shape defects contained in the learning data set and the reconstructed shape data, such as the features of shapes such as perforations 202 and spatter 204. Therefore, when the reconstructed shape data is input into the learned determination model, it is possible to reliably and accurately determine whether the shape of the welding part 201 is good or not.

[0293] In addition, according to the present embodiment, it is possible to generate a required amount of learning data set based on a small amount of sample shape data, and to improve the accuracy of the determination model. Therefore, it is possible to accurately determine whether the shape of the welding part 201 is good or not. In addition, it is not necessary to obtain a large amount of sample shape data for learning, and the man-hours required for determining whether the shape is good or not can be significantly reduced. In addition, since the learning data set is generated after classifying the sample shape data in advance according to each material of the workpiece 200 or according to each material and shape of the workpiece 200, it is possible to efficiently generate the learning data set.

[0294] In addition, the learning data set generation unit 26 generates a learning data set based on one or more feature amounts extracted from the sample shape data.

[0295] By generating a learning data set using the feature amounts extracted from the sample shape data, it is possible to simplify the generation process of the learning data set without reducing the accuracy of the determination model.

[0296] The learning data set generation unit 26 performs data augmentation processing by changing one or more feature amounts extracted from the sample shape data, or changing the position of the shape defect part in the sample shape data, or performing both.

[0297] By generating a learning data set based on one or more feature amounts extracted from the sample shape data, it is possible to improve the production efficiency of the learning data and further reduce the man-hours. In addition, by simple processing such as changing the feature amounts, the position, size, number, etc. of the shape defects, it is possible to efficiently generate the learning data set.

[0298] (Other embodiments)

[0299] In Figure 1 In the example shown, an example in which both the welding torch 11 (welding head 11) and the shape measurement unit 21 are mounted on the robot 16 is shown, but another robot (not shown) on which the shape measurement unit 21 is mounted may be provided separately from the robot 16 on which the welding torch 11 (welding head 11) is mounted. In this case, various data are sent from another robot control unit (not shown) that controls the operation of the other robot to the data processing unit 23.

[0300] In addition, the learning dataset generation unit 26 shown in the second embodiment classifies the sample shape data according to each material and shape of the workpiece 200, and performs data augmentation processing on the classified sample shape data to generate a plurality of learning datasets.

[0301] However, the learning dataset generation unit 26 may not have this classification function. In this case, the determination model generation unit 27 may also not have the function of generating a determination model according to each material and shape of the workpiece 200.

[0302] -Industrial Applicability-

[0303] When the appearance inspection device of the present disclosure includes a curved portion at the welding portion, it is also possible to accurately evaluate the three-dimensional shape of the welding portion, so it is particularly useful in the appearance inspection of workpieces including various welding portions.

[0304] -Symbol Explanation-

[0305] 10 Welding device

[0306] 11 Welding head (torch)

[0307] 12 Welding wire

[0308] 13 Welding wire feeding device

[0309] 14 Power supply

[0310] 15 Output control unit

[0311] 16 Robot

[0312] 17 Robot control unit

[0313] 20 Appearance inspection device

[0314] 21 Shape measurement unit

[0315] 22 Sensor control unit

[0316] 23 Data processing unit

[0317] 24 Shape data processing unit

[0318] 25 First storage unit

[0319] 26 Learning dataset generation unit

[0320] 27 Determination model generation unit

[0321] 28 First determination unit

[0322] 29 Notification unit

[0323] 100 Welding system

[0324] 200 workpieces

[0325] 201 welding part.

Claims

1. An appearance inspection device for inspecting the appearance of a welded part of a workpiece, characterized in that: The appearance inspection device at least includes a shape measurement unit and a data processing unit. The shape measurement unit is mounted on a robot and measures the three-dimensional shape of the welded part along the welding line. The data processing unit processes the shape data obtained by the shape measurement unit. The data processing unit at least includes a shape data processing unit and a first determination unit. The shape data processing unit reconstructs the shape data obtained by the shape measurement unit. The first determination unit determines whether the shape of the welded part is good or not based on the shape data reconstructed by the shape data processing unit and one or more prepared determination models. In the plane where the welded part is formed, when the direction orthogonal to the welding line is set as the X direction, the direction extending along the welding line is set as the Y direction, and the direction orthogonal to the X direction and the Y direction respectively is set as the Z direction, The shape measurement unit measures the shape of the welded part at multiple points along the X direction and measures the shape of the welded part at multiple points along the Y direction. In the case where the welded part includes a curved portion, the shape data processing unit corrects the coordinate points of the shape data located at different positions along the X direction so that the Y-direction resolution of the shape data is the same, and based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, corrects the coordinate position in the Z direction of the shape data, thereby reconstructing the shape data.

2. The appearance inspection device according to claim 1, characterized in that: In the case where the welded part has a specified width in the X direction, the shape data processing unit corrects the Y-direction resolution of the shape data so that the Y-direction resolution of the shape data is the same as the Y-direction resolution on the trajectory of the central position in the X direction passing through the welded part.

3. The appearance inspection device according to claim 1, characterized in that: The determination model is strengthened by learning using a prepared learning data set. The learning data set includes qualified product data and learning data. The qualified product data is the shape data that does not include shape defects in the welded part. The learning data is data for which the type of the shape defect is determined and marked for the shape data that includes the shape defect in the welded part, i.e., unqualified product data. The first determination unit inputs the shape data input from the shape data processing unit into the determination model. The determination model determines the presence or absence of the shape defect, and determines the type, number, size of the shape defect, and the position of the shape defect relative to the welded part. Based on each determination result and determination result, it determines whether the shape of the welded part is good or not.

4. The appearance inspection device according to claim 3, characterized in that: The appearance inspection device further includes a learning data set generation unit and a determination model generation unit. The learning dataset generation unit performs data augmentation processing on a plurality of sample shape data previously obtained by the shape measurement unit to generate a plurality of the learning datasets. The determination model generation unit uses the plurality of learning datasets to generate the determination model.

5. The appearance inspection device according to claim 4, characterized in that: The learning dataset generation unit classifies at least the plurality of sample shape data obtained by the shape measurement unit according to each material of the workpiece, and performs data augmentation processing on the classified sample shape data to generate a plurality of the learning datasets. The determination model generation unit uses the plurality of learning datasets to generate the determination model at least according to each material of the workpiece.

6. The appearance inspection device according to claim 1, characterized in that: The data processing unit further includes a notification unit, and the notification unit notifies the determination result in the first determination unit.

7. A welding system, characterized in that: The welding system includes: The appearance inspection device according to claim 1; and A welding device for welding the workpiece, The welding device at least includes a welding head and an output control unit, The welding head is used to input heat to the workpiece, The output control unit controls the welding output of the welding head.

8. The welding system according to claim 7, characterized in that: The welding device further includes the robot and a robot control unit, The robot holds the welding head and moves the welding head to a desired position, The robot control unit controls the actions of the robot. When the first determination unit determines that the shape of the welding part is defective, the output control unit stops the welding output of the welding head, the robot control unit stops the actions of the robot, or makes the robot work in such a way that the welding head comes to a specified initial position.

9. An appearance inspection method for a welding part, which uses the appearance inspection device according to any one of claims 1 to 6, characterized in that: The appearance inspection method for the welding part at least includes: A first step, in which the shape measurement unit measures the three-dimensional shape of the welding part while moving together with the robot to obtain the shape data. A second step, in which the shape data processing unit reconstructs the shape data. A third step, in which the first determination unit determines whether the shape of the welding part is good or bad based on the shape data reconstructed by the shape data processing unit and one or more determination models prepared in advance. In the second step, when the welding part includes a curved portion, the shape data processing unit corrects the coordinate points of the shape data located at different positions along the X direction so that the Y-direction resolution of the shape data is the same, and based on the coordinate position in the X direction and the corrected coordinate position in the Y direction, corrects the coordinate position in the Z direction of the shape data, thereby reconstructing the shape data.

10. The appearance inspection method for the welded part according to claim 9, wherein: When the welded part has a specified width in the X direction, in the second step, the shape data processing unit corrects the Y-direction resolution of the shape data so that the Y-direction resolution of the shape data is the same as the Y-direction resolution on the trajectory of the center position in the X direction of the welded part.

11. The appearance inspection method for the welded part according to claim 9, wherein: The third step includes: A step in which the first determination unit determines whether there is a shape defect in the input shape data; When there is the shape defect in the shape data, a step of determining the number, size, and position of the shape defect in a specified area of the welded part and its surroundings; and A step of determining the type of the shape defect.

Images