Appearance inspection device, welding system, shape data correction method, and appearance inspection method for welding portion

Abstract

The appearance inspection device (20) includes: a shape measuring unit (21) for measuring the three-dimensional shape of the welded part (201); and a data processing unit (23) for processing the shape data obtained by the shape measuring unit (21). The data processing unit (23) includes: a shape data processing unit (24) for resolving the shape data; a learning dataset generation unit (26) for generating multiple learning datasets by performing data augmentation processing on multiple sample shape data obtained in advance; a judgment model generation unit (27) for generating a judgment model using multiple learning datasets; and a first judgment unit (28) for determining whether the shape of the welded part (201) is good or bad based on the resolving shape data and the judgment model.

Description

Technical Field

[0001] This disclosure relates to a visual inspection device, a welding system, a method for correcting shape data, and a method for visual inspection of welded parts. Background Technology

[0002] In recent years, the method of using judgment models enhanced through machine learning to perform visual inspections of welded parts and other components to determine the quality of their shapes has been widely used.

[0003] For example, Patent Document 1 proposes a visual inspection device for welded parts, which includes a shape measurement unit, an image processing unit, a learning dataset generation unit, a judgment model generation unit, and a first judgment unit.

[0004] The shape measurement unit measures the shape of the weld joint, and the image processing unit generates image data of the weld joint based on the measured shape data. The learning dataset generation unit classifies multiple image data sets according to each material and shape of the workpiece and performs data augmentation to generate multiple learning datasets. The judgment model generation unit uses the multiple learning datasets to generate a judgment model for the shape of the weld joint according to each material and shape of the workpiece. The first judgment unit determines the quality of the weld joint shape based on the image data read from the image processing unit and the judgment model.

[0005] Patent Document 1: International Publication No. 2020 / 129617 Summary of the Invention

[0006] -The technical problem the invention aims to solve-

[0007] However, in actual workpiece production and processing sites, the inspection conditions of visual inspection devices will be appropriately modified. For example, the inspection speed when inspecting welded parts along the welding line, the measurement frequency of the sensor, and the measurement resolution of the sensor will be changed accordingly to achieve the conditions most suitable for the user of the processing equipment.

[0008] The reason for changing the inspection conditions in this way is that the production cycle time and inspection accuracy of the workpiece vary significantly depending on the inspection conditions. For example, when production cycle time is critical, increasing the inspection speed allows for inspection within a shorter production cycle time. On the other hand, because the sensor's measurement resolution becomes coarser, the three-dimensional shape of the welded area obtained by the sensor also becomes coarser. Therefore, minute weld defects cannot be detected, resulting in reduced inspection accuracy.

[0009] Furthermore, while prioritizing inspection accuracy, the measurement resolution is refined by slowing down the inspection speed. On the other hand, the slower inspection speed results in a longer production cycle time.

[0010] Furthermore, inspection conditions vary depending on the workpiece being inspected and the condition of the welded parts. For example, the inspection speed may vary depending on whether the workpiece is curved or straight. Additionally, high-precision inspection is required for machined parts where even minor defects are unacceptable. In such cases, it is necessary to increase measurement resolution, even at the cost of sacrificing production cycle time.

[0011] In addition, the materials and shapes of the workpieces to be inspected are diverse, and the inspection conditions need to be adjusted to suit the shape of each welded part.

[0012] However, for example, if the sensor's measurement resolution changes, the measurement results will differ even when measuring the three-dimensional shape of the same welded area. Therefore, the features of the welded area shape data input into the judgment model are inconsistent with the features of the shape data after prior machine learning, making high-precision visual inspection impossible.

[0013] This disclosure was made to solve the above-mentioned technical problems, and its purpose is to provide a visual inspection device, welding system, shape data correction method, and visual inspection method for welded parts that can accurately evaluate the three-dimensional shape of the welded parts even when the inspection conditions change.

[0014] -Technical solutions for solving technical problems-

[0015] To achieve the above objectives, the appearance inspection device disclosed herein is an appearance inspection device for inspecting the appearance of welded parts of a workpiece. The appearance inspection device is characterized by: the appearance inspection device comprising at least a shape measurement unit and a data processing unit; the shape measurement unit is mounted on a robot and measures the three-dimensional shape of the welded part along the welding line; the data processing unit processes the shape data obtained by the shape measurement unit; the data processing unit comprises at least a shape data processing unit, a learning dataset generation unit, a judgment model generation unit, and a first judgment unit; the shape data processing unit performs at least resolution correction processing on the shape data obtained by the shape measurement unit; the learning dataset generation unit performs data augmentation processing on multiple sample shape data pre-obtained by the shape measurement unit to generate multiple learning datasets; the judgment model generation unit uses the multiple learning datasets to generate a judgment model for determining the goodness or badness of the shape of the welded part; and the first judgment unit determines the goodness or badness of the shape of the welded part based on the shape data corrected by the shape data processing unit and one or more judgment models generated by the judgment model generation unit.

[0016] The welding system disclosed herein is characterized in that: the welding system includes the visual inspection device and a welding device for welding the workpiece, the welding device including at least a welding head and an output control unit, the welding head being used to input heat to the workpiece, and the output control unit controlling the welding output of the welding head.

[0017] The shape data correction method disclosed herein is a method for correcting shape data obtained by an appearance inspection device. The method is characterized by comprising: a step whereby the shape measurement unit measures the three-dimensional shape of the welded portion while moving with the robot, and obtains sample shape data for generating multiple learning datasets; a step whereby the shape measurement unit measures the three-dimensional shape of the welded portion while moving with the robot, and obtains the shape data; and a step whereby, if the resolution of the shape data obtained by the shape measurement unit is different from the resolution of the sample shape data, the shape data processing unit corrects the shape data so that the resolution of the shape data obtained by the shape measurement unit becomes equal to the resolution of the sample shape data.

[0018] The visual inspection method for welded parts disclosed herein uses a visual inspection device. The method comprises at least the following steps: a shape measurement unit measures the three-dimensional shape of the welded part while moving with the robot, obtaining sample shape data for generating multiple learning datasets; a judgment model generation unit uses the multiple learning datasets to generate one or more judgment models for judging the goodness or badness of the welded part's shape; the shape measurement unit measures the three-dimensional shape of the welded part while moving with the robot, obtaining the shape data; if the resolution of the shape data obtained by the shape measurement unit differs from the resolution of the sample shape data, the shape data processing unit corrects the shape data so that the resolution of the shape data obtained by the shape measurement unit becomes equal to the resolution of the sample shape data; and a first judgment unit determines the goodness or badness of the welded part's shape based on the shape data with the resolution corrected by the shape data processing unit and one or more judgment models generated by the judgment model generation unit.

[0019] -The Effects of the Invention-

[0020] According to this disclosure, even when inspection conditions change, the three-dimensional shape of the welded part can be evaluated with good accuracy. Furthermore, the quality of the welded part's shape can be accurately determined. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the structure of the welding system involved in the embodiment;

[0022] Figure 2 This is a schematic diagram showing the hardware structure of the robot processing unit;

[0023] Figure 3 This is a functional block diagram of the visual inspection device;

[0024] Figure 4 This is a schematic diagram illustrating the shape measurement of the weld seam performed by the shape measuring unit;

[0025] Figure 5A This is a schematic diagram showing the hardware structure of the sensor control unit;

[0026] Figure 5B This is a schematic diagram showing the hardware structure of the data processing unit;

[0027] Figure 6A This is a top view schematic diagram illustrating an example of a defective pattern at a welded joint;

[0028] Figure 6B It is along Figure 6A A schematic diagram of a cross-section cut along the VIB-VIB line;

[0029] Figure 6C It is along Figure 6A A schematic diagram of a cross-section cut along the VIC-VIC line;

[0030] Figure 6D It is along Figure 6A A cross-sectional diagram of the VID-VID line;

[0031] Figure 6E It is along Figure 6A A schematic diagram of a cross-section cut along the VIE-VIE line;

[0032] Figure 7A This is a schematic diagram illustrating an example of the steps involved in creating a learning dataset;

[0033] Figure 7B This is a diagram illustrating another example of the steps involved in creating a learning dataset;

[0034] Figure 7C This is another illustration showing the steps involved in creating a learning dataset;

[0035] Figure 8A This is a flowchart of the visual inspection steps for welded areas;

[0036] Figure 8B This is a flowchart of the steps for determining the quality of the weld shape;

[0037] Figure 9 This is a conceptual diagram illustrating the steps for deriving coordinate points of shape data during resolution conversion correction;

[0038] Figure 10 This is a conceptual diagram illustrating the changes in the coordinate positions of shape data before and after resolution conversion correction within the acceleration / deceleration interval;

[0039] Figure 11 This is a schematic diagram illustrating an example of a robot's speed control function during acceleration and deceleration.

[0040] Figure 12 This is a schematic diagram illustrating the visual inspection of the welded area involved in Embodiment 1;

[0041] Figure 13 This is a schematic diagram showing the top view shape of the welding parts involved in Embodiments 2 and 3;

[0042] Figure 14 This is a schematic diagram illustrating the visual inspection of the welded area involved in Embodiment 4;

[0043] Figure 15 This is a schematic diagram showing the Z-direction profile of the welded portion involved in Embodiment 5;

[0044] Figure 16A This is a schematic diagram showing the arrangement of the shape measuring units in interval A;

[0045] Figure 16B This is a schematic diagram showing the arrangement of the shape measuring units in interval B;

[0046] Figure 16C This is a schematic diagram showing the arrangement of the shape measuring units in interval C;

[0047] Figure 17A This is a schematic diagram showing the arrangement of the shape measuring unit involved in the conventional method, that is, the arrangement when checking interval A;

[0048] Figure 17B This is a schematic diagram showing the arrangement of the shape measuring unit involved in the conventional method, that is, the arrangement when checking interval B. Detailed Implementation

[0049] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. It should be noted that the following description of preferred embodiments is merely illustrative and is not intended to limit the present disclosure, its application, or its uses.

[0050] (Implementation Method)

[0051] [Structure of the welding system]

[0052] Figure 1 This is a schematic diagram showing the structure of the welding system involved in this embodiment. The welding system 100 includes a welding apparatus 10 and a visual inspection apparatus 20.

[0053] The welding apparatus 10 includes a welding torch 11, a wire feed device 13, a power supply 14, an output control unit 15, a robot 16, and a robot control unit 17. By supplying power from the power supply 14 to the welding wire 12 held in the welding torch 11, an electric arc is generated between the tip of the welding wire 12 and the workpiece 200, and the workpiece 200 is subjected to arc welding by the input of heat. It should be noted that the welding apparatus 10 includes other components and equipment such as pipes and gas cylinders for supplying shielding gas to the welding torch 11, but for ease of explanation, their illustrations and descriptions are omitted. It should be noted that the power supply 14 is also referred to as a welding power source.

[0054] The output control unit 15 is connected to the power supply 14 and the welding wire feed device 13. The output control unit 15 controls the welding output of the welding torch 11 according to the prescribed welding conditions; in other words, it controls the power supplied to the welding wire 12 and the power supply time. Furthermore, the output control unit 15 controls the feed speed and feed amount of the welding wire 12 fed from the welding wire feed device 13 to the welding torch 11. It should be noted that the welding conditions can be directly input to the output control unit 15 via an input unit (not shown), or they can be selected from a welding program read from another recording medium, etc.

[0055] Robot 16 is a known multi-joint robot that holds welding torch 11 at its front end and is connected to robot control unit 17. Robot control unit 17 controls the movement of robot 16, causing the front end of welding torch 11, or in other words, the front end of welding wire 12 held on welding torch 11, to move along a predetermined welding trajectory to the desired position.

[0056] Figure 2 The schematic diagram shows the hardware structure of the robot processing unit. The robot control unit 17 includes at least a CPU (Central Processing Unit) 17a, a driver IC (Driver Integrated Circuit) 17b, a RAM (Random Access Memory) 17c, and an IC (Integrated Circuit) 17d.

[0057] During normal operation, IC 17d receives output signals from rotation detectors (not shown) located on multiple joint axes of the robot 16. These output signals are processed by IC 17d and then input to CPU 17a. CPU 17a sends control signals to drive IC 17b based on the signals input from IC 17d and the rotational speeds of the joint axes set in a predefined program stored in RAM 17c. Drive IC 17b, based on the control signals from CPU 17a, controls the rotation of servo motors (not shown) connected to the joint axes.

[0058] Furthermore, as described later, in the appearance inspection device 20, if the first determination unit 28 of the data processing unit 23 determines that the shape of the welded part 201 is defective, the CPU 17a of the robot control unit 17, which receives the determination result, stops the operation of the robot 16, or makes the robot 16 work in such a way that the welding torch 11 comes to the predetermined initial position.

[0059] Furthermore, the output control unit 15 has the same structure as the robot control unit 17. That is, the output control unit 15 includes at least a CPU 15a, a driver IC 15b, a RAM 15c, and an IC 15d.

[0060] In normal operation, IC 15d receives a signal corresponding to the output of power supply 14. This signal is processed by IC 15d and then input to CPU 15a. CPU 15a, based on the signal input from IC 15d and the output of power supply 14 set in a predefined program stored in RAM 15c, sends a control signal to driver IC 15b. Driver IC 15b, based on the control signal from CPU 15a, controls the output of power supply 14, thereby controlling the welding output of welding torch 11.

[0061] Furthermore, as described later, in the visual inspection device 20, if the first determination unit 28 determines that the shape of the welded part 201 is defective, the CPU 15a of the output control unit 15, which receives the determination result, stops the output of the power supply 14. This stops the welding output of the welding torch 11.

[0062] It should be noted that both the output control unit 15 and the robot control unit 17 may include Figure 2 Other components not shown. For example, as a storage device, it may also include ROM (Read Only Memory) and HDD (Hard Disk Drive).

[0063] The appearance inspection device 20 includes a shape measuring unit 21, a sensor control unit 22, and a data processing unit 23. The shape measuring unit 21 is mounted on the robot 16 or the welding torch 11 and measures the shape of the welding part 201 of the workpiece 200. The structure of the appearance inspection device 20 will be described in detail below.

[0064] It should be noted that, in Figure 1 In this example, an arc welding apparatus for performing arc welding is shown as welding apparatus 10, but it is not particularly limited to this. For example, welding apparatus 10 could also be a laser welding apparatus for performing laser welding. In this case, instead of welding torch 11, a laser head (not shown) connected to a laser oscillator (not shown) via an optical fiber (not shown) is mounted and held on robot 16. Furthermore, in the following description, welding torch 11 and laser head are sometimes collectively referred to as welding head 11.

[0065] [Structure of the visual inspection device]

[0066] Figure 3 This is a functional block diagram of the visual inspection device. Figure 4 This is a schematic diagram showing the shape measurement of the weld seam performed by the shape measuring unit. Figure 5A A schematic diagram showing the hardware structure of the sensor control unit is provided. Figure 5B A schematic diagram showing the hardware structure of the data processing unit.

[0067] The shape measuring unit 21 is, for example, a laser light source 21a (see reference). Figure 4 ), and camera (not shown) or light sensor matrix 21b (see reference) Figure 4 The three-dimensional shape measuring sensor is composed of a laser light source 21a configured to scan the surface of the workpiece 200, and the camera or light-receiving sensor matrix 21b captures the reflection trajectory (hereinafter sometimes referred to as the shape line) of the laser beam projected onto the surface of the workpiece 200.

[0068] like Figure 4 As shown, the shape measuring unit 21 scans a predetermined area including the welding part 201 and its surroundings using a laser (emitted light). The light-receiving sensor matrix 21b captures the reflected light after the emitted light is reflected from the surface of the workpiece 200, thereby measuring the three-dimensional shape of the welding part 201. It should be noted that the welding part 201 is a so-called weld seam formed along a direction that extends along a welding line predetermined by a welding procedure, etc. It should be noted that in the following description, the direction extending along this welding line will sometimes be referred to as the Y direction (see reference...). Figure 6AOn the other hand, the direction orthogonal to the Y direction on the surface of the workpiece 200 where the welded portion 201 is formed is sometimes referred to as the X direction. Additionally, the height direction of the welded portion 201 when the surface of the workpiece 200 is used as a reference is sometimes referred to as the Z direction. The Z direction is orthogonal to both the X and Y directions.

[0069] It should be noted that in this application specification, "orthogonal," "parallel," or "identical" refers to the meaning of "orthogonal," "parallel," or "identical," including the manufacturing tolerances, assembly tolerances, machining tolerances of the components constituting the welding system 100, and deviations in the moving speed of the robot 16. It does not imply that the objects being compared are strictly orthogonal, parallel, or identical to each other.

[0070] by Figure 4 In the example shown, the emitted light from the laser source 21a illuminates multiple points along the width direction of the welding area 201, which in this case is along the X direction. The laser light illuminating the multiple points is reflected, and the reflected light is captured by the light-receiving sensor matrix 21b. In addition, the shape measuring unit 21, which is held in the robot 16, moves along the Y direction at a predetermined speed, and illuminates the welding area 201 and its surroundings with emitted light at predetermined time intervals during the movement, and the reflected light is captured by the light-receiving sensor matrix 21b each time.

[0071] It should be noted that, as described above, the shape measuring unit 21 is configured to not only measure the shape of the welded portion 201, but also measure the shape of its surrounding area within a specified range. This is to evaluate the presence or absence of spatter 204 and contaminants 206 (see below). Figure 6A ).

[0072] Here, "measurement resolution" refers to the distance between adjacent measurement points in the shape data measured by the shape measurement unit 21. For example, the measurement resolution in the X direction is the distance between adjacent measurement points along the X direction. The measurement resolution in the X direction is set based on the performance of the shape measurement unit 21, mainly the performance of the light-receiving sensor matrix 21b, and more specifically, the X-direction dimensions of each sensor included in the light-receiving sensor matrix 21b and the distance between the sensors.

[0073] The measurement resolution is set in the X, Y, and Z directions respectively. As will be described later, the measurement resolution in the Y direction varies depending on the moving speed of the robot 16 or the sampling frequency of the light sensor matrix 21b.

[0074] Furthermore, when simply referred to as "resolution," this resolution refers to the interval between adjacent coordinate points in the multiple point cloud data of the welding portion 201 acquired by the shape measurement unit 21. As described later, shape data is reconstructed based on the shape of the welding portion 201. The resolution of the shape data before reconstruction is equal to the measurement resolution described above. On the other hand, the resolution of the reconstructed shape data is sometimes different from the measurement resolution. In the example shown in this application specification, the X-direction resolution of the shape data is equal to the X-direction measurement resolution. On the other hand, the Y-direction resolution of the shape data is sometimes different from the Y-direction measurement resolution. Resolutions are set in the X, Y, and Z directions respectively.

[0075] like Figure 5A As shown, the sensor control unit 22 includes at least a CPU 22a and a RAM 22b. In the sensor control unit 22, the CPU 22a sends control commands to the shape measurement unit 21 to control the operation of the shape measurement unit 21. The control commands sent from the CPU 22a to the shape measurement unit 21 include, for example, the check conditions of the shape measurement unit 21, measurement start commands, and measurement stop commands. The RAM 22b stores pre-set check conditions. It should be noted that other data may also be stored therein. Additionally, the sensor control unit 22 may also include... Figure 5A Other components not shown. For example, as a storage device, it may also include ROM or HDD.

[0076] The data processing unit 23 receives the point cloud data of the shape lines obtained by the shape measurement unit 21 as shape data and processes it.

[0077] In addition, such as Figure 3 As shown, the data processing unit 23 is composed of multiple functional blocks. Specifically, the data processing unit 23 includes a shape data processing unit 24, a first storage unit 25, a learning dataset generation unit 26, a decision model generation unit 27, a first decision unit 28, and a notification unit 29.

[0078] like Figure 5B As shown, the data processing unit 23 includes at least a CPU 23a, a GPU (Graphics Processing Unit) 23b, RAM / ROM 23c, an IC 23d, an input port 23e, an output port 23f, and a data bus 23h as hardware. The data processing unit 23 also includes a display 23g.

[0079] Figure 5B The hardware structure of the data processing unit 23 shown is the same as that of a known personal computer (PC). Furthermore, Figure 3 The multiple function blocks within the data processing unit 23 shown are processed by... Figure 5BThis is achieved by executing specified software in the various devices shown, especially CPU 23a and GPU 23b. It should be noted that, although... Figure 5B Examples of various devices connected to a single data bus 23h are shown, but multiple data buses can also be set up, depending on the purpose, just like a typical PC.

[0080] In the data processing unit 23, the shape data processing unit 24 has a noise removal function for the shape data acquired by the shape measuring unit 21. Since the reflectivity of the laser beam emitted from the shape measuring unit 21 varies depending on the material of the workpiece 200, excessively high reflectivity can cause halos and other noise, sometimes affecting the shape data. Therefore, the shape data processing unit 24 is configured to perform noise filtering via software. It should be noted that noise can also be removed by setting an optical filter (not shown) on the shape measuring unit 21 itself. By combining the optical filter and the software filtering, high-quality shape data can be obtained. Furthermore, this improves the quality of the judgment model for the learning dataset described later, enabling high-precision determination of the shape of the welded part 201.

[0081] The noise removal function of the shape data processing unit 24 is mainly implemented by the IC 23d of the data processing unit 23. However, it is not particularly limited to this; for example, the GPU 23b of the data processing unit 23 can also perform noise removal on the shape data.

[0082] The shape data processing unit 24 performs statistical processing on the point cloud data to correct any tilting or deformation of the base portion of the welding part 201 relative to a predetermined reference surface, such as the mounting surface of the workpiece 200. In addition, for example, to emphasize the shape and position of the welding part 201, edge emphasis correction is sometimes performed to emphasize the periphery of the welding part 201.

[0083] Furthermore, the shape data processing unit 24 extracts feature quantities of the shape data based on the shape of the workpiece 200 and the inspection items related to the shape of the welded portion 201. In this case, for each shape data, one or more feature quantities corresponding to one or more inspection items are extracted. Additionally, the extracted feature quantities are associated with the shape data for subsequent data processing. Here, feature quantities refer to various specific factors extracted from the shape data. Representative feature quantities include the length, width, and height of the welded portion 201 from the reference plane, as well as the differences in length, width, and height between multiple points within the welded portion 201. However, it is not particularly limited to these; feature quantities are appropriately set according to the content determined in each inspection item.

[0084] Furthermore, the shape data processing unit 24 is configured to perform resolution conversion correction on the acquired shape data. The resolution conversion correction of shape data will be described in detail later.

[0085] The edge enhancement correction processing function, feature extraction function, and resolution conversion correction function of the shape data processing unit 24 are mainly implemented by the CPU 23a of the data processing unit 23. However, it is not particularly limited to this; for example, some or all of the edge enhancement correction processing may also be performed by the IC 23d or the GPU 23b.

[0086] The first storage unit 25 stores shape data of the weld portions 201 in other workpieces 200 that have been processed before welding the workpiece 200 to be evaluated. Additionally, the first storage unit 25 stores shape data obtained experimentally before welding the actual workpiece 200. In the following description, this pre-obtained shape data will sometimes be referred to as sample shape data.

[0087] The sample shape data includes qualified product data (those with good shape) and unqualified product data (those with certain shape defects) of the welded parts 201 to be evaluated. The unqualified product data was processed by changing the number and location of shape defects to create a state marked with multiple shape defects; the processing results were used as multiple training datasets. The marked unqualified product data and qualified product data were summarized as the training dataset before data augmentation processing. It should be noted that the shape data of the welded parts 201 in other workpieces 200 and the shape data of the welded parts 201 in the workpiece 200 to be evaluated were obtained specifically for the same welded parts 201 within workpieces 200 with the same shape and material.

[0088] Furthermore, the inspection conditions of the shape measuring unit 21 are fixed when obtaining sample shape data. However, the inspection conditions can be changed according to each material of the workpiece 200 or each shape of the workpiece 200.

[0089] The learning dataset generation unit 26 reads the sample shape data generated by the shape data processing unit 24 and stored in the first storage unit 25, and classifies the sample shape data according to the material and shape of the workpiece 200. Alternatively, it can classify the data according to each inspection item of the welded part 201. In this case, the same shape data can be included in different inspection items. Furthermore, the learning dataset generation unit 26 generates a learning dataset based on the feature quantities associated with the sample shape data, according to each material and shape of the workpiece 200; that is, it generates a set of learning data that is input into the judgment model described later and used to improve the judgment accuracy of the judgment model. For example, the material and shape of the workpiece 200 are organized in matrix form to determine the classification category, and the learning dataset is classified accordingly (see reference). Figure 3 It should be noted that, as examples of the shape of workpiece 200, examples include butt joints or overlaps of sheet metal, T-joints, or cross joints.

[0090] Furthermore, the learning dataset generation unit 26 performs data augmentation processing on the sample shape data read from the first storage unit 25 to generate a learning dataset. Specifically, data augmentation processing is performed by changing one or more feature quantities associated with the sample shape data, changing the position of the shape defective parts in the sample shape data of the welding part 201, or performing both. The steps for generating the learning dataset will be described in detail later.

[0091] The functions of the learning dataset generation unit 26 are primarily implemented by the CPU 23a of the data processing unit 23. However, it is not particularly limited to this; for example, a portion of the functions may also be implemented by the GPU 23b.

[0092] The judgment model generation unit 27 generates a judgment model based on the judgment criteria set for each inspection item of the welded part 201 according to each material and shape of the workpiece 200. The generated judgment model is represented by a combination of multiple weighted recognizers. The judgment model is, for example, a known object detection algorithm represented by a CNN (Convolutional Neural Network).

[0093] Furthermore, the decision model generation unit 27 inputs multiple learning datasets corresponding to each material and shape of the workpiece 200 into each decision model generated according to each material and shape of the workpiece 200, and repeatedly performs learning, thereby improving the decision accuracy of each decision model. In this case, for example, according to Figure 3 The classification categories shown are used to generate the decision model. It should be noted that the learning process is repeated until the accuracy rate, reproducibility rate, and precision of the decision model meet the preset values.

[0094] Furthermore, when generating the judgment model, the qualified and unqualified product data from the sample shape data are appropriately selected and used based on the material and shape of the workpiece 200. This shortens the generation time of the judgment model and achieves high accuracy. Similarly, when generating the judgment model according to each inspection item of the welded part 201, the qualified and unqualified product data from the sample shape data are appropriately selected and used based on the inspection item. This shortens the generation time of the judgment model and achieves high accuracy.

[0095] The first determination unit 28 determines whether the shape of the welded part 201 is good based on the shape data of the welded part 201 that has been processed by the shape data processing unit 24 for noise removal, edge emphasis, etc., and the determination model corresponding to the selected inspection item in the determination model generated by the determination model generation unit 27. In other words, it determines whether the prescribed determination criteria are met.

[0096] It should be noted that before determining the shape quality of welded part 201, a learning dataset was used to strengthen the judgment model through learning. Specifically, for the selected inspection items, the learning dataset generated by the learning dataset generation unit 26 is input into the judgment model generated by the judgment model generation unit 27. The judgment results are then manually confirmed by welding operators or others. If the type of welding defect does not match the learning data, annotation is performed. It should be noted that annotation refers to the process of marking the presence and type of shape defects determined by visual observation of the actual welded part 201 at the corresponding parts of the shape data. This annotation is basically done manually.

[0097] By performing annotations, the presence and type of shape defects are re-evaluated in the training data. Based on the results, the training dataset is remade or newly created, and then the decision model is relearned using the annotated training dataset. By performing these tasks one or more times, the decision model is strengthened through learning.

[0098] Additionally, for the training dataset, as described later, resolution correction is performed beforehand as needed. By performing appropriate resolution correction, the training dataset can be used to strengthen the decision model through learning.

[0099] However, as will be discussed later, since there are multiple patterns of shape defects, the actual method used is to calculate the probability of the shape defects contained in the shape data. If the probability exceeds a certain threshold, a shape defect is determined to exist, and its type is identified. This will be discussed in detail later.

[0100] For example, the consistency between the types of shape defects annotated in the learning data and the types of shape defects contained in the shape data of the welded part 201 is determined by probability. If the probability exceeds a predetermined threshold, the types of shape defects contained in the shape data of the welded part 201 are determined.

[0101] The first determination unit 28 outputs the following information: whether a shape defect exists, and if a shape defect exists, the type, number, size, and location of the shape defect at the welded part 201. Furthermore, if a shape defect exceeds a threshold based on a pre-set determination criterion, the unit outputs whether the shape of the welded part 201 is good or bad. It should be noted that this threshold varies depending on the type and size of the shape defect. For example, for spatter described later (see...) Figure 6A If five or more spatter particles with a diameter of 5 μm or larger are present, the shape of weld 201 is deemed defective. Regarding perforations (see...) Figure 6A , 6C If more than one of these conditions exists, the shape of welded part 201 is deemed defective. It should be noted that these are just examples and can be appropriately modified based on the aforementioned criteria and thresholds.

[0102] It should be noted that the threshold and display format for determining shape defects can be arbitrarily set. For example, it could be: if it is a splatter 204, it would be displayed in red; if it is a perforation 202 (see...). Figure 6A If the presence or absence of spatter 204 and the upper limit of spatter 204 are set as inspection items, the portion identified as spatter 204 can be displayed in a different color from the background, and the probability of it being spatter 204 can also be distinguished by color. Therefore, welding operators or system administrators can easily identify the presence or distribution of this defective part at a glance. For example, if the probability of the above-mentioned uniformity is less than 30%, it can be displayed in green; if the probability is greater than 70%, it can be displayed in red. It should be noted that the probability range and corresponding color settings can be arbitrarily set in this case.

[0103] It should be noted that since there are multiple inspection items for the shape of the welded part 201, the goodness or badness of the shape is judged according to each inspection item. The final qualified product is judged only when all the inspection items that need to be judged have been met.

[0104] The notification unit 29 is configured to notify the output control unit 15, the robot control unit 17, the welding operator, or the system administrator of the determination result from the first determination unit 28. For example, the display 23g of the data processing unit 23 corresponds to the notification unit 29. The determination result can be displayed on the display 23g or a display unit (not shown) of the welding system 100, or output to a printer (not shown), or notification can be made through both. If only the final determination result is notified, it can also be output as sound from a sound output unit (not shown). It should be noted that the determination result notified from the notification unit 29 preferably includes not only the final determination result but also the determination result for each inspection item. In this way, the welding operator or system administrator can specifically know what kind of defect has occurred at the welding position 201.

[0105] It should be noted that the configuration is such that when the determination result in the first determination unit 28 is positive, that is, when it is determined that the shape of the welding part 201 is good, the welding system 100 continuously welds the next welding part 201 in the same workpiece 200, or welds the same welding part 201 in the next workpiece 200.

[0106] On the other hand, if the determination result in the first determination unit 28 is negative, that is, if the shape of the welding part 201 is determined to be defective, the output control unit 15 stops the welding output of the welding torch 11, and the robot control unit 17 stops the operation of the robot 16, or makes the robot 16 work in such a way that the welding torch 11 comes to the predetermined initial position.

[0107] [Steps for generating the learning dataset]

[0108] Figures 6A-6E This illustrates an example of a shape defect that occurs at the welded area. Figures 7A to 7C This illustrates an example of the steps involved in generating a learning dataset. It should be noted that... Figures 6A-6E The shape of the welded part 201 during butt welding is shown. Figure 6A Showing the top view shape, Figures 6B to 6E Show along Figure 6A A sectional view cut along the VIB-VIB line to the VIE-VIE line.

[0109] like Figures 6A-6EAs shown, when workpiece 200 is subjected to arc welding or laser welding, various shape defects may occur at the welded area 201 due to poor welding conditions or the use of low-quality workpiece 200. For example, sometimes a portion of the welded area 201 may burn through (hereinafter, a through hole formed in workpiece 200 by a portion of the welded area 201 burning through is sometimes referred to as a perforation 202) or undercut 203 may occur. It should be noted that undercut 203 refers to a defective portion where the edge of the weld is recessed from the surface of workpiece 200. In addition, the variation in the length, width, and height from the reference surface of the welded area 201 may sometimes exceed the allowable range ΔL, ΔW, ΔH compared to their respective design values ​​L, W, and H. Additionally, sometimes spatter 204 is generated when a molten droplet (not shown) formed at the tip of the welding wire 12 is transferred to the workpiece 200, or when part of the molten droplet or particles of molten metal from the workpiece 200 are scattered. Or, if the workpiece 200 is a galvanized steel sheet, a portion of the molten droplet evaporates from the welding area 201, resulting in a pit 205. Or, if the workpiece 200 or the welding wire 12 is an aluminum-based material, dirt 206 is generated near the welding area 201.

[0110] It should be noted that pit 205 is the opening on the surface of the weld, and contaminant 206 is a black carbon black-like deposit that is generated near the weld. It includes the aforementioned perforation 202, undercut 203, spatter 204, etc., and is one of the patterns (types) of shape defects.

[0111] Thus, various patterns of shape defects exist in the welded area 201, requiring separate judgment criteria for each. For example, for perforations 202 and undercuts 203, it is necessary not only to judge their presence or absence but also to establish criteria for identifying them, such as contrast or height difference with the surrounding area of ​​the welded area 201. Furthermore, for spatter 204, its average diameter needs to be set, and its quality is judged based on the number of spatter 204 with an average diameter exceeding a specified value per unit area. Moreover, the number of inspection items or the judgment criteria for the shape of the welded area 201 can be changed or added depending on the material of the workpiece 200, the welded area, and customer specifications.

[0112] Furthermore, the criteria for determining whether there is a shape defect based on shape data vary depending on the material and shape of the workpiece 200. As mentioned above, the reflectivity of the laser beam varies depending on the material of the workpiece 200; therefore, for example, the brightness level and contrast of the shape data will also change. In addition, depending on the shape of the workpiece 200, even when welding straight sections of the same length, the weld shape of the welded portion 201 may change due to the influence of gravity, etc.

[0113] Therefore, the judgment model generation unit 27 needs to generate a judgment model using a large amount of learning data for each material and shape of the workpiece 200. That is, it needs to obtain a large amount of shape data of the welding parts 201 suitable as learning data for each material and shape of the workpiece 200. However, obtaining the required sample shape data in advance for each material and shape of the workpiece 200 requires a huge amount of time, resulting in low efficiency.

[0114] Therefore, in this embodiment, in the learning dataset generation unit 26, the sample shape data read from the first storage unit 25 is classified according to each material and shape of the workpiece 200, and data augmentation processing is performed on the classified sample shape data to generate multiple groups of learning data required when generating the judgment model, i.e., learning datasets.

[0115] For example, such as Figure 7A As shown, as a learning dataset, multiple data points are generated by varying the length or position of the welded portion 201, which is one of the feature quantities, within the original sample shape data. It should be noted that... Figure 7A The example shown is a number of shape data that are shortened beyond the allowable range ΔL compared to the reference value L of the length of the welded part 201, but it is not particularly limited to this. In addition, shape data that are lengthened beyond the allowable range ΔL compared to the reference value L of the length are also generated.

[0116] Or, such as Figure 7B As shown, as a learning dataset, multiple data points are generated by varying the size or position of the perforation 202 within the original sample shape data. In this case, as feature quantities, the height from the reference plane and the difference in height between multiple points within the welded area 201 are extracted and varied.

[0117] Or, such as Figure 7C As shown, as a learning dataset, multiple datasets are generated by varying the number or position of the splashes 204 in the original sample shape data.

[0118] In addition, the same feature quantity is extracted around the welded part 201, and a learning dataset is generated based on the feature quantity. Thus, it is possible to determine whether the spatter 204 and the dirt 206 exceed the specified allowable range.

[0119] [Visual inspection steps for welded areas]

[0120] Figure 8A A flowchart showing the steps for visual inspection of welded areas is provided. Figure 8B A flowchart showing the steps for determining the quality of a weld shape. Figure 9This is a conceptual diagram illustrating the steps for deriving coordinate points of shape data during resolution conversion correction. Figure 10 This is a conceptual diagram showing the changes in the coordinate positions of shape data before and after resolution conversion correction in the acceleration / deceleration interval. Figure 11 This is a schematic diagram illustrating an example of a robot's speed control function during acceleration and deceleration.

[0121] When performing a visual inspection of the welded part 201 using a pre-prepared learning dataset, the inspection conditions of the shape measuring unit 21 must be consistent with the following conditions, which are the same as the inspection conditions when obtaining the shape data used to generate the learning data, i.e., when obtaining the sample shape data.

[0122] On the other hand, as mentioned earlier, the inspection conditions often change depending on the production cycle time and inspection accuracy of the workpiece 200. However, in this case, the shape data of the welded part 201 and the feature quantities extracted from the shape data change according to the inspection conditions, which may prevent the first determination unit 28 from correctly determining whether the shape of the welded part 201 is good or bad.

[0123] Therefore, in this embodiment, instead of making the two inspection conditions identical, the resolution of the measured shape data is corrected. One inspection condition is the condition used when obtaining pre-acquired sample shape data, and the other inspection condition is the condition used when obtaining the shape data to be measured. Specifically, considering the measurement resolution of the shape measuring unit 21, the shape data is corrected so that the resolution of the measured shape data reaches a value equal to the resolution of the pre-acquired sample shape data. This correction is the resolution conversion correction described above. In this way, even if the inspection conditions change, the first determination unit 28 can accurately determine whether the shape of the welded part 201 is good or bad. This will be further explained below.

[0124] First, the shape of the welding part 201 is measured by the shape measuring unit 21. Figure 8A Step S1) is to obtain shape data.

[0125] Next, the data processing unit 23 obtains the moving speed of the robot 16 from the robot control unit 17, that is, the scanning speed of the shape measuring unit 21 in the Y direction of the welding part 201. Based on the moving speed of the robot 16, the data processing unit 23 further divides the scanning range of the shape measuring unit 21 in the Y direction into a constant speed range and an acceleration / deceleration range. Figure 8AStep S2). Here, "constant velocity range" refers to the range in which the shape measuring unit 21 installed on the robot 16 moves at a constant velocity along the Y direction. "Acceleration / deceleration range" refers to the range in which the shape measuring unit 21 installed on the robot 16 accelerates and / or decelerates along the Y direction.

[0126] The shape data processing unit 24 performs the aforementioned edge enhancement correction, noise removal, and other processing on the shape data in the selected interval (hereinafter referred to as the selected interval) from the intervals segmented in step S2. Figure 8A Step S3).

[0127] Next, the data processing unit 23 determines whether the selected interval in step S3 is a constant speed interval. This determination is made by checking whether the speed control function of the robot 16 within the selected interval is constant speed, that is, whether it is a constant speed relative to time. Figure 8A (Step S4). The speed control function of robot 16 is sent from robot control unit 17 to data processing unit 23 according to the request of data processing unit 23.

[0128] <Case where the selected interval is a constant speed interval>

[0129] If the determination result in step S4 is affirmative, i.e., the selected interval is a constant speed interval, the shape data processing unit 24 calculates the X-direction resolution (hereinafter referred to as X-direction resolution) and Y-direction resolution (hereinafter referred to as Y-direction resolution) of the shape data obtained in step S1, and stores them in the first storage unit 25. Figure 8A Step S5).

[0130] As described above, the X-direction resolution is equivalent to the distance between adjacent measurement points along the X-direction. It should be noted that, generally, the scanning width of the laser beam of the shape measuring unit 21 is constant. Furthermore, the Y-direction resolution in the constant velocity range is expressed by the following equation (1).

[0131] Ry=1000V / 60F=50V / 3F……(1)

[0132] Here, Ry(mm) is the Y-direction resolution of the shape data in the selected interval, V(m / min) is the moving speed of robot 16, and F(Hz) is the measurement frequency of shape measuring unit 21.

[0133] That is, in the Y direction, the shape is measured at a period of 1 / F. On the other hand, in the X direction, multiple measurement points are measured together at a period of 1 / F within the scanning width of the laser beam. It should be noted that, generally, the X-direction resolution of the shape data is determined based on the pixel size and spacing of the camera or light-receiving sensor matrix 21b (not shown) in the shape measurement unit 21. This is also the case when the selected interval is a constant speed interval or an acceleration / deceleration interval.

[0134] According to the request of the data processing unit 23, the sensor control unit 22 sends the X-direction resolution and measurement frequency F of the shape measurement unit 21 to the data processing unit 23.

[0135] After executing step S5, the shape data processing unit 24 determines whether the X-direction resolution and Y-direction resolution of the shape data in the selected interval are equal to the X-direction resolution and Y-direction resolution of the sample shape data that have been acquired and stored in the first storage unit 25 in advance. Figure 8A Step S6).

[0136] If the judgment result in step S6 is positive, proceed to step S8, where the first determination unit 28 determines whether the shape of the welding part 201 in the selected range is good or bad. The details of step S8 will be explained below.

[0137] On the other hand, if the judgment result in step S6 is negative, the shape data processing unit 24 performs resolution conversion correction on the shape data in the selected range. Figure 8A Step S7).

[0138] like Figure 9 As shown, the resolution conversion correction is performed by using the height in the Z direction of each of the adjacent coordinate points (x, y) at the specified coordinate point (x, y). Here, Rx (mm) is the X-direction resolution of the shape data in the selected interval.

[0139] It should be noted that, in the following description, the height (Z coordinate) of the coordinate point (x, y) in the Z direction will be set as Z(x, y). It should also be noted that the origin of the coordinate system containing the coordinate point (x, y) is, for example, set at the beginning of the welding section 201. In this case, the origin of the Z coordinate Z(x, y) is set with the surface of the workpiece 200 near this beginning as a reference.

[0140] Resolution conversion correction is performed through the following steps. First, as shown in equations (2) and (3), the resolution coefficient Cx in the X direction and the resolution coefficient Cy in the Y direction are calculated respectively.

[0141] Cx=Rx / Rx0……(2)

[0142] Cy=Ry / Ry0……(3)

[0143] Here, Rx0 is the X-direction resolution of the sample shape data, and Ry0 is the Y-direction resolution of the sample shape data. The X-direction resolution Rx0 and the Y-direction resolution Ry0 are pre-stored in the first storage unit 25.

[0144] Next, for each point whose XY coordinates are reconstructed at the resolution when the sample shape data is obtained, Z(Xn / Cx, Ym / Cy) is calculated in a way that satisfies Equation (4) to be used as the Z coordinate.

[0145] Z(Xn / Cx, Ym / Cy)=(1-dx)×(1-dy)×Z(x, y)+dx×(1-dy)×Z(x+Rx, y)+(1-dx)×dy×Z(x, y+Ry)+dx×dy×Z(x+Rx, y+Ry)……(4)

[0146] Here, n is the variable corresponding to each point in the point cloud data along the X direction. n is an integer, satisfying the relationship 1≤n≤N (N is the number of points in the X direction). m is the variable corresponding to each point in the point cloud data along the Y direction. m is an integer, satisfying 1≤m≤M (M is the number of points in the Y direction).

[0147] Here, dx is the distance in the X direction between coordinates (x, y) and (Xn / Cx, Ym / Cy) divided by the distance in the X direction between coordinates (x, y) and (x+Rx, y). Similarly, dy is the distance in the Y direction between coordinates (x, y) and (Xn / Cx, Ym / Cy) divided by the distance in the Y direction between coordinates (x, y) and (x, y+Ry).

[0148] Right now, Figure 9 The ratio of dx to (1-dx) is the ratio of the distance in the X direction between coordinate points (x, y) and (Xn / Cx, Ym / Cy) to the distance in the X direction between coordinate points (x+Rx, y) and (Xn / Cx, Ym / Cy). Similarly, the ratio of dy to (1-dy) is the ratio of the distance in the Y direction between coordinate points (x, y) and (Xn / Cx, Ym / Cy) to the distance in the Y direction between coordinate points (x, y+Ry) and (Xn / Cx, Ym / Cy).

[0149] The height of the coordinate point (Xn / Cx, Ym / Cy) in the Z direction is derived based on the height of the four points surrounding the coordinate point (Xn / Cx, Ym / Cy), namely the original coordinate points (x, y), (x+Rx, y), (x, y+Ry), and (x+Rx, y+Ry).

[0150] For all point clouds included in the selected interval, calculate Z(Xn / Cx, Ym / Cy) as the corrected coordinates shown in equation (4), and the resolution conversion correction of the shape data is completed.

[0151] After performing step S7, step S8 is performed, and the shape data after resolution conversion correction is used to determine whether the shape of the welding part 201 in the selected range is good or bad through the first determination unit 28.

[0152] After executing step S8, the data processing unit 23 determines whether there are any intervals in the segmented intervals of the shape data that were not processed in step S3. Figure 8A Step S9).

[0153] If the judgment result of step S9 is positive, return to step S3, select the interval where the processing of step S3 was not performed, and perform the processing of step S3. Repeat a series of steps until the judgment result of step S9 is negative.

[0154] If the judgment result of step S9 is negative, then there is no remaining interval where the measured weld shape data has not undergone noise removal or other preprocessing, and there is no segmented interval where shape evaluation has not been performed. Therefore, the visual inspection of weld 201 ends.

[0155] <Case where the selected interval is an acceleration / deceleration interval>

[0156] If the judgment result in step S4 is negative, that is, if the selected interval is an acceleration / deceleration interval, the data processing unit 23 obtains the X-direction resolution of the shape data obtained in step S1 from the sensor control unit 22 and stores it in the first storage unit 25. Figure 8A Step S10).

[0157] Next, the shape data processing unit 24 calculates the Y-direction resolution of the shape data obtained in step S1 based on the speed control function of the robot 16. Figure 8A (Step S11). According to the request of the data processing unit 23, the speed control function of the robot 16 is sent from the robot control unit 17 to the shape data processing unit 24 of the data processing unit 23. It should be noted that the speed control function of the robot 16 can also be temporarily stored in the first storage unit 25 and then sent to the shape data processing unit 24.

[0158] Furthermore, the shape data processing unit 24 performs resolution conversion correction on the shape data in the selected range.

[0159] Here, refer to Figure 10 The resolution of shape data during acceleration and deceleration, especially the resolution in the Y direction, is explained. Figure 8A Step S11).

[0160] Typically, when performing visual inspection on a welded area 201, the scanning frequency and width of the laser beam are rarely changed. As described above, when the Y direction is set as the direction extending along the weld line, the X direction is the direction intersecting the direction extending along the weld line. The laser beam of the shape measuring unit 21, used for measuring the shape, scans along the Y direction at the moving speed of the front end of the robot 16 (hereinafter referred to as the moving speed of the robot 16) while periodically scanning across the weld line. Therefore, in each interval of the constant speed interval and the acceleration / deceleration interval, the X-direction resolution Rx of the shape data is mostly considered constant.

[0161] On the other hand, the Y-direction resolution Ry varies depending on the moving speed V of the robot 16. When the selected interval is a constant speed interval, since the measurement frequency F and the moving speed V are constant, it can be seen from equation (1) that the Y-direction resolution Ry is also constant.

[0162] On the other hand, when the selected interval is an acceleration / deceleration interval, for example, when robot 16 accelerates, the Y-direction interval between adjacent measurement points widens over time. Conversely, when robot 16 decelerates, the Y-direction interval between adjacent measurement points narrows over time. As a result, for example, as... Figure 10 As shown on the left, the resolution in the Y direction varies between measurement points along the Y direction. As mentioned earlier, when the shape of the welded part 201 is evaluated based on such point cloud data (shape data), accurate results cannot be obtained.

[0163] Therefore, when the selected interval is the acceleration / deceleration interval, the Y-direction resolution needs to be corrected to the form corresponding to the speed control function of robot 16. Specifically, the Y-direction resolution Ry(t) (mm) is expressed as shown in equation (5).

[0164] Ry(t)=1000V(t) / 60F=(50 / 3F)×V(t)……(5)

[0165] Here, V(t) (m / min) is the speed control function of robot 16, and F (Hz) is the measurement frequency of shape measuring unit 21. As will be described later, V(t) is described as a k-th power of time t (sec) (k is an integer greater than or equal to 1).

[0166] In addition, the resolution coefficient Cym(t) at the m-th coordinate point arranged along the Y direction from the origin is expressed as shown in equation (6).

[0167] [Mathematical Expression 1]

[0168]

[0169] Here, Tm is the time taken to move from the origin to the m-th coordinate point.

[0170] Next, for each point whose XY coordinates are reconstructed at the resolution when the sample shape data is obtained, the Z coordinate as the corrected coordinate is calculated in a manner that satisfies Equation (7).

[0171] Z(Xn / Cx, Ym / Cy) m (t))=(1-dx)×(1-dv)×Z(x, y)+dx×(1-dy)×Z(x+Rx, y)+(1-dx)×dy×Z(x, y+Ry)+dx×dy×Z(x+Rx, y+Ry)……(7)

[0172] In addition to reconstructing the Y-coordinate Ym / Cy m Except that (t) is described as a function of time t, equation (7) is the same as equation (4).

[0173] For all point clouds contained in the selected interval, calculate Z(Xn / Cx, Ym / Cy) as shown in equation (7). m (t)), the resolution conversion and correction of the shape data is then completed. Figure 8A Step S12).

[0174] After performing step S12, the process proceeds to step S13, where the shape data, after resolution conversion correction, is used by the first determination unit 28 to determine whether the shape of the welding portion 201 in the selected range is good or bad. The details of step S13 will then be explained.

[0175] After performing step S13, the data processing unit 23 determines whether there are any intervals in the segmented intervals of the shape data that were not processed in step S3. Figure 8A Step S9).

[0176] When the determination result in step S9 is affirmative, return to step S3, select an interval where the processing of step S3 has not been performed, and perform the processing of step S3. Repeat a series of steps until the determination result in step S9 is negative.

[0177] If the determination result in step S9 is negative, since there is no divided interval where shape evaluation has not been performed, the visual inspection of the welded part 201 is terminated.

[0178] Here, as Figure 11 shown, consider the case where the shape data is divided into three intervals (interval 1 to interval 3).

[0179] From Figure 11 it can be seen that interval 1 and interval 3 are constant-speed intervals. Therefore, the steps S1 to S8 of Figure 8A are respectively performed to conduct a visual inspection of the welded part 201.

[0180] On the other hand, interval 2 is a deceleration interval. Specifically, during the period T (sec), the moving speed V (m / min) of the robot 16 monotonically decreases from V1 to V2 (<V1). Therefore, the speed control function V(t) of the robot 16 in interval 2 is expressed in the form shown in Equation (8).

[0181] V(t) = A×t + B = ((V2 - V1) / T)×t + V1......(8)

[0182] That is, the speed control function V(t) is a linear function of time t, the linear coefficient A of time t is (V2 - V1) / T, and the constant B is V1.

[0183] In this case, the data processing unit 23 obtains various information representing the speed control function V(t) from the robot control unit 17. For example, when the speed control function V(t) is a k-th order function of time t (k is an integer greater than or equal to 1), the coefficient values of t to t k and the value of the constant B are obtained. In addition, in interval 2, the steps S1 to S4 and S10 to S13 of Figure 8A are respectively performed to conduct a visual inspection of the welded part 201.

[0184] <Determination step for the quality of the weld shape>

[0185] Figure 8B The determination step for the quality of the weld shape (welded part 201) shown in Figure 8A is common in step S8 and step S13 of

[0186] Figure 8A Therefore, they will be described together. Step S8 and step S13 of Figure 8B The sub-steps SA to SC are shown. First, the first determination unit 28 determines whether the shape data in the selected interval contains shape defects (step SA). As described above, the determination model used in this case is a determination model that has been strengthened by learning in advance using a learning dataset. When strengthening by learning, the resolution of the learning dataset can also be corrected in advance.

[0187] In addition, the first determination unit 28 determines the size, number, and location of the shape defect in the welding part 201 (step SB). Furthermore, the first determination unit 28 determines the type of shape defect (step SC).

[0188] It should be noted that in step SC, as described above, the type of defect is determined by considering the shape, size, and location of the defect at the welded part 201. In this case, for example, the probability that the defect is spatter 204 is calculated, and if the probability is a specified value (e.g., 70%) or higher, the defect is determined to be spatter 204.

[0189] It should be noted that the final determination result of whether the shape of the welded part 201 is good or bad is sent to the notification unit 29 and the display 23g. In addition, if the determination result is bad, the shape data obtained by the shape measurement unit 21, specifically the shape of the welded part 201, is displayed as a point cloud data on the display 23g.

[0190] In addition, if the result of the quality assessment of all welded parts 201 included in a workpiece 200 is good, the workpiece 200 is determined to be a qualified product and is sent to the subsequent processing steps, or is shipped out as a qualified product.

[0191] On the other hand, if a defect is found in one or more welded parts 201 of a workpiece 200, several countermeasures can be taken. For example, after visually inspecting all welded parts 201 of the workpiece 200, the inspection results are saved, and the workpiece 200 is discarded as a defective product. In this case, the inspection results are saved, for example, in the first storage unit 25 of the data processing unit 23. However, it is not particularly limited to this. Alternatively, the workpiece 200 can be discarded as a defective product as soon as a defect in a welded part 201 is discovered.

[0192] Alternatively, for example, after visually inspecting all welded parts 201 included in workpiece 200, the inspection results can be saved, and workpiece 200 can be sent to the repair process. In the repair process, the welded parts 201 that are determined to be defective are re-welded.

[0193] Alternatively, for example, after visually inspecting all welded parts 201 included in workpiece 200, the inspection results can be saved, and the welding operator can visually confirm again the welded parts 201 that have been determined to be defective. Through visual confirmation, it can be determined whether the welded part 201 can be repaired. If it is determined that it can be repaired, the workpiece 200 is moved to the repair process, and the welded parts 201 that have been determined to be defective are re-welded.

[0194] [Effects, etc.]

[0195] As described above, the appearance inspection device 20 of this embodiment inspects the appearance of the welded portion 201 of the workpiece 200.

[0196] The appearance inspection device 20 includes at least a shape measuring unit 21 and a data processing unit 23. The shape measuring unit 21 is mounted on the robot 16 and measures the three-dimensional shape of the welding part 201 along the welding line. The data processing unit 23 processes the shape data obtained by the shape measuring unit 21.

[0197] The data processing unit 23 includes at least a shape data processing unit 24, which performs at least resolution correction processing on the shape data. Additionally, the data processing unit 23 includes a learning dataset generation unit 26 and a judgment model generation unit 27. The learning dataset generation unit 26 performs data augmentation processing on multiple sample shape data pre-acquired by the shape measurement unit 21 to generate multiple learning datasets. The judgment model generation unit 27 uses the multiple learning datasets to generate a judgment model for determining the quality of the shape of the welded part 201.

[0198] The data processing unit 23 also includes a first determination unit 28, which determines whether the shape of the welded part 201 is good or bad based on the shape data corrected by the shape data processing unit 24 and one or more determination models generated by the determination model generation unit 27.

[0199] By configuring the appearance inspection device 20 in this way, even if the inspection conditions change due to changes in production cycle time or required inspection accuracy, the three-dimensional shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape of the welded part 201 can be correctly determined.

[0200] In addition, a workpiece 200 typically contains multiple welded parts 201. In this case, there are usually multiple welded parts 201 with different shapes, and the inspection conditions will be changed accordingly depending on the shape of the welded parts 201.

[0201] According to this embodiment, even when a workpiece 200 contains multiple welding parts 201 with different inspection conditions, the three-dimensional shape of each welding part 201 can be evaluated with good accuracy, and the goodness or badness of the shape of the welding part 201 can be correctly determined.

[0202] The shape data processing unit 24 corrects the resolution of the shape data obtained by the shape measurement unit 21 based on the inspection conditions of the shape measurement unit 21. The inspection conditions include, for example, the measurement resolution, measurement frequency, and scanning speed of the shape measurement unit 21. It should be noted that, as described above, the scanning speed of the shape measurement unit 21 corresponds to the scanning speed in the X direction of the laser beam, the moving speed V of the robot 16, or the speed control function V(t) of the robot 16.

[0203] In this way, the resolution conversion and correction of shape data can be performed easily and with good accuracy.

[0204] Sample shape data is acquired under conditions where the measurement resolution, measurement frequency, and scanning speed of the shape measuring unit 21 are preset. The shape data processing unit 24 corrects the resolution of the shape data acquired by the shape measuring unit 21 to make it equal to the resolution of the sample shape data.

[0205] As described above, the judgment model is reinforced by learning from each of the multiple learning datasets. Multiple learning datasets are generated based on sample shape data, which is pre-obtained experimentally before welding the actual workpiece 200. The resolution of the shape data is adjusted to be equal to the resolution of the sample shape data. In this way, the shape features of the defective parts contained in the sample shape data, and consequently the defective parts contained in the learning datasets and the shape data themselves, such as perforations 202 and spatter 204, can be matched. Therefore, the goodness or badness of the shape of the welded part 201 using the learned judgment model can be reliably and accurately determined.

[0206] The appearance inspection device 20 also includes a sensor control unit 22, which stores the inspection conditions of the shape measurement unit 21 and sends the stored inspection conditions to the data processing unit 23. When the direction extending along the weld line is set as the Y direction, the sensor control unit 22 sends the measurement resolution and measurement frequency of the X direction, which intersects the Y direction and the height direction of the welded part 201, i.e., the Z direction, to the data processing unit 23.

[0207] In addition, the data processing unit 23 receives the moving speed V of the robot 16 or the speed control function V(t) of the robot 16 from the robot control unit 17 that controls the movement of the robot 16.

[0208] In this way, the resolution conversion and correction of shape data can be performed easily and with good accuracy.

[0209] The shape data processing unit 24 corrects the Z-direction value of the shape data based on the X-direction resolution and Y-direction resolution of the shape data.

[0210] While the robot 16 moves at a constant speed along the welding line, the shape measuring unit 21 measures the three-dimensional shape of the welding part 201, and the Y-direction resolution is determined based on the measurement frequency of the shape measuring unit 21 and the moving speed V of the robot 16.

[0211] While the robot 16 accelerates and / or decelerates within a predetermined range extending along the welding line, the shape measurement unit 21 measures the three-dimensional shape of the welding portion 201. The Y-direction resolution is determined based on the measurement frequency F of the shape measurement unit 21 and the speed control function V(t) of the robot 16. The speed control function V(t) is described as a function of degree k with respect to time t. However, it is not limited to this; for example, the speed control function V(t) can also be a sine wave function or a cosine wave function. That is, the speed control function V(t) is a function dependent on time t.

[0212] In this way, even when the scanning speed of the shape measuring unit 21 is varied, the resolution conversion and correction of the shape data can be performed easily and with good accuracy.

[0213] The learning dataset generation unit 26 classifies multiple sample shape data pre-obtained by the shape measurement unit 21 according to each material and shape of the workpiece 200, and performs data augmentation processing on the classified sample shape data to generate multiple learning datasets.

[0214] The decision model generation unit 27 uses multiple learning datasets to generate decision models according to each material and shape of the workpiece 200.

[0215] By configuring the appearance inspection device 20 in this way, even with a small amount of sample shape data, a sufficient amount of learning dataset can be generated, resulting in a highly accurate judgment model. Therefore, it is possible to accurately determine whether the shape of the welded part 201 is good or bad. Furthermore, the time spent determining shape quality is significantly reduced by eliminating the need to acquire a large amount of sample shape data. Additionally, shape defects in the welded part 201 can be automatically detected without manually setting complex judgment criteria. Moreover, since the learning dataset is generated after pre-classifying the sample shape data according to each material and shape of the workpiece 200, the learning dataset can be generated efficiently.

[0216] The data processing unit 23 also includes a first storage unit 25, which stores at least sample shape data for generating multiple learning datasets. In this case, the sample shape data stored in the first storage unit 25 is read out to the learning dataset generation unit 26 to generate multiple learning datasets.

[0217] In this way, the generation of the learning dataset and the subsequent generation and processing of the decision model can be carried out smoothly.

[0218] In addition, the data processing unit 23 also includes a notification unit 29, which notifies the first determination unit 28 of the determination result.

[0219] In this way, during the welding of workpiece 200, welding operators or system administrators can be aware in real time of whether any defects have occurred at the welding location 201. Furthermore, they can take appropriate measures, such as deciding whether to continue welding workpiece 200, as needed. This reduces welding costs in the welding process.

[0220] The learning dataset generation unit 26 generates a learning dataset based on one or more features extracted from the sample shape data. It should be noted that the features are extracted by the shape data processing unit 24.

[0221] By using features extracted from sample shape data to generate the learning dataset, the generation process of the learning dataset can be simplified without reducing the accuracy of the decision model.

[0222] The learning dataset generation unit 26 performs data augmentation by changing one or more feature quantities extracted from the sample shape data, changing the position of the poorly shaped parts in the sample shape data, or performing both.

[0223] By generating a learning dataset based on one or more features extracted from sample shape data, the efficiency of learning data production can be improved, further reducing working hours. Furthermore, by simply changing the features or the location of shape defects, the learning dataset can be generated efficiently.

[0224] It should be noted that the learning dataset generation unit 26 can also classify multiple sample shape data according to each inspection item of the welding part 201, and perform data augmentation processing on the classified sample shape data to generate multiple learning datasets.

[0225] When determining whether the shape of the welded part 201 is good or bad, the first determination unit 28 determines whether there is a shape defect in the input shape data. During this determination, a training dataset is generated using sample shape data, wherein the sample shape data consists of qualified product data that does not contain shape defects and unqualified product data that contains a certain type of shape defect. In the training dataset, the unqualified product data is processed to determine the type of shape defect and to label the determined type of shape defect. Using this training dataset, the determination model is strengthened in advance through learning.

[0226] In addition, if there are shape defects in the shape data, the first determination unit 28 determines the number and size of the shape defects, as well as the location of the shape defects in the welded part 201 and the specified area around it.

[0227] Furthermore, the first determination unit 28 determines the type of shape defect. This determination is made by referring to the number, size, or location of the shape defect at the welded portion 201. Additionally, the type of shape defect is determined through probability calculation; if the probability is above a predetermined threshold, the type of shape defect is determined. It should be noted that the type of shape defect is not limited to... Figures 6A-6E The defects shown are as follows. The situation where the dimensions of welded part 201 do not meet the specified acceptable product standards is also included in the shape defects category. The acceptable product standards for this dimension can be set in any of the X, Y, or Z directions.

[0228] As described above, the first determination unit 28 determines or identifies multiple items regarding the shape of the welded portion 201, and based on these results, ultimately determines whether the shape of the welded portion 201 is good or bad. In this way, the goodness or badness of the shape of the welded portion 201 can be evaluated correctly and reliably.

[0229] In addition, the judgment model generation unit 27 can also use multiple learning datasets to generate a judgment model for judging whether the shape of the welded part 201 is good or bad, according to each inspection item of the welded part 201.

[0230] Furthermore, the learning dataset generation unit 26 can also divide multiple sample shape data into a specific part and other parts, perform data augmentation processing on each sample shape data separately, and generate multiple learning datasets. The aforementioned specific part is the part where the shape defect of the welded part 201 is difficult to determine compared with other parts.

[0231] In addition, when using multiple learning datasets and generating a determination model according to each material and shape of the workpiece 200, the determination model generation unit 27 can generate a determination model corresponding to a specific part of the welding part 201 and a determination model corresponding to other parts.

[0232] In this way, even in specific parts of the welded area 201 where it is difficult to determine the presence and type of shape defects compared to other parts, the presence and type of shape defects can be determined with an accuracy exceeding a specified value. As a result, it is possible to determine with good accuracy whether the welded area 201 is in good shape during visual inspection.

[0233] It should be noted that in this embodiment, the determination model used to determine whether there is a shape defect is the same as the determination model used to determine the type of shape defect. However, it is also possible to set separate determination models.

[0234] The welding system 100 involved in this embodiment includes a welding device 10 for welding workpieces 200 and a visual inspection device 20.

[0235] By configuring the welding system 100 in this way, the shape of the welded part 201 can be inspected with good accuracy in a shorter amount of time. As a result, the cost of the welding process can be reduced.

[0236] In addition, the welding apparatus 10 includes at least a welding head 11 (welding torch 11), a robot 16, an output control unit 15, and a robot control unit 17. The welding head 11 (welding torch 11) is used to input heat to the workpiece 200. The robot 16 holds the welding head 11 (welding torch 11) and moves the welding head 11 (welding torch 11) to the desired position. The output control unit 15 controls the welding output of the welding head 11 (welding torch 11), and the robot control unit 17 controls the movement of the robot 16.

[0237] If the first determination unit 28 of the visual inspection device 20 determines that the shape of the welded part 201 is defective, the output control unit 15 stops the welding output of the welding head 11 (welding torch 11), and the robot control unit 17 stops the operation of the robot 16, or makes the robot 16 work in such a way that the welding head 11 (welding torch 11) comes to the predetermined initial position.

[0238] By configuring the welding system 100 in this way, if the shape of the welded part 201 is defective, subsequent welding can be stopped, preventing frequent occurrences of defective products. It should be noted that by obtaining the judgment result in the first judgment unit 28 according to each inspection item, the location of the defect in the welding system 100 can be deduced, the cause of the defect can be quickly eliminated, and the downtime of the welding system 100 can be shortened.

[0239] The shape data correction method according to this embodiment includes the following steps: the shape measurement unit 21 measures the three-dimensional shape of the welding part 201 while moving together with the robot 16, and obtains sample shape data for generating multiple learning datasets. Additionally, it includes the following step: the shape measurement unit 21 measures the three-dimensional shape of the welding part 201 while moving together with the robot 16, and obtains shape data.

[0240] The shape data correction method further includes the following steps: when the resolution of the shape data obtained by the shape measuring unit 21 is different from the resolution of the sample shape data, the shape data processing unit 24 corrects the shape data so that the resolution of the shape data obtained by the shape measuring unit 21 becomes equal to the resolution of the sample shape data.

[0241] In this way, the shape defects contained in the sample shape data can be matched with the shape features of the shape defects contained in the training dataset and the shape data, such as the perforation 202 and the spatter 204. Therefore, when the resolution-converted and corrected shape data is input into the learned judgment model, the goodness or badness of the shape of the welded part 201 can be reliably and accurately determined.

[0242] The appearance inspection method of the welded part 201 involved in this embodiment includes at least the following steps: the shape measuring unit 21 measures the three-dimensional shape of the welded part 201 while moving together with the robot 16, and obtains sample shape data for generating multiple learning datasets.

[0243] The process also includes the following steps: the judgment model generation unit 27 uses multiple learning datasets to generate one or more judgment models for judging whether the shape of the welded part 201 is good or bad.

[0244] In addition, the following steps are included: the shape measuring unit 21 measures the three-dimensional shape of the welding part 201 while moving together with the robot 16, and obtains shape data.

[0245] The visual inspection method for the welded part 201 also includes the following steps: when the resolution of the shape data obtained by the shape measuring unit 21 is different from the resolution of the sample shape data, the shape data processing unit 24 corrects the shape data so that the resolution of the shape data obtained by the shape measuring unit 21 becomes equal to the resolution of the sample shape data.

[0246] In addition, the following steps are included: the first determination unit 28 determines whether the shape of the welded part 201 is good or bad based on the shape data whose resolution has been corrected by the shape data processing unit 24 and one or more determination models generated by the determination model generation unit 27.

[0247] According to this embodiment, even when the inspection conditions change due to factors such as production cycle time and required inspection accuracy, the three-dimensional shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape of the welded part 201 can be correctly determined.

[0248] Furthermore, according to this embodiment, when performing visual inspection on multiple welded portions 201 included in a workpiece 200, even if the inspection conditions are changed according to the shape of each welded portion 201 each time, the shape of each welded portion 201 can be evaluated with good accuracy. For example, even if the range of motion of the robot 16 is limited due to the presence of bent or recessed portions in the welded portions 201, the shape of the welded portions 201 can be evaluated with good accuracy regardless of changes in the inspection conditions.

[0249] Furthermore, for welding operators, inspection conditions can be set separately for parts that require high-speed inspection due to production cycle time considerations and parts that require low-speed inspection due to emphasis on inspection accuracy. For example, such inspection conditions can be set within a single welded part 201, or among multiple welded parts 201 included in a workpiece 200. Even so, according to this embodiment, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0250] In addition, the first determination unit 28 determines whether the shape of the welded part 201 is good or bad. Figure 8A Steps S8 and S13 also include the following sub-steps.

[0251] That is, based on the shape data input from the shape data processing unit 24 and the judgment model that has been strengthened through learning, a sub-step is performed to determine whether there is a shape defect in the welded part 201. Figure 8B Sub-step SA); Sub-step to determine the number, size, and position of the defects relative to weld 201. Figure 8B The sub-step SB); and the sub-step for determining the type of shape defect ( Figure 8B Sub-step SC).

[0252] The first determination unit 28 determines whether the shape of the welded part 201 is good or bad based on the determination results and confirmation results of each sub-step SA to SC.

[0253] In this way, the shape of the welded part 201 can be evaluated correctly and reliably.

[0254] It should be noted that during the visual inspection of the welded part 201, the decision to set an acceleration / deceleration zone is made based on the shape of the welded part 201, the required production cycle time, etc., and the movement speed of the robot 16 is also determined. Therefore, when performing visual inspection on the welded part 201, it is necessary to appropriately perform resolution conversion and correction of the shape data in the shape data processing unit 24 based on the shape of the welded part 201, the required production cycle time, etc. This will be explained below through an embodiment.

[0255] Example 1

[0256] Figure 12 This is a schematic diagram illustrating the visual inspection of the welded area according to Embodiment 1. It should be noted that, for ease of explanation, the shape of the shape measuring unit 21 is shown schematically. Furthermore, illustrations and explanations of components other than the shape measuring unit 21 and the welded area 201 are omitted. Additionally, in Figure 12 In the following figures, the same reference numerals are used for the same parts as in the first embodiment, and detailed descriptions are omitted.

[0257] exist Figure 12 In the example shown, the shape measuring unit 21 was moved at a constant speed relative to the straight weld seam, i.e., the welded portion 201, while visual inspection of the straight weld seam, i.e., the welded portion 201, was performed. The measuring range (scanning range) of the shape measuring unit 21 is 5 mm in the X direction and 45 mm in the Y direction.

[0258] Furthermore, the measurement frequency F is set to 1000Hz, the moving speed V of robot 16 (equivalent to the constant speed mentioned above) is set to 9m / min, and the measurement resolution in the X direction of shape measuring unit 21 is set to 0.05mm (=50μm). In this case, the X-direction resolution Rx of the shape data is also 0.05mm (=50μm). In addition, substituting the values ​​of the measurement frequency F and the moving speed V mentioned above into equation (1), the Y-direction resolution Ry of the shape data is 0.15mm (=150μm).

[0259] In addition, the X-direction resolution Rx0 of the sample shape data is 0.025 mm (=25 μm), and the Y-direction resolution Ry0 is 0.1 mm (=100 μm). Therefore, in order to perform resolution conversion correction, the following steps are performed. Substitute each value into equations (2) and (3) respectively, so that the X-direction resolution coefficient Cx=2 and the Y-direction resolution coefficient Cy=1.5.

[0260] In addition, when the number of columns (number of columns) of the point cloud data measured by the shape measuring unit 21 is set to Nx in the X direction and Ny in the Y direction, and the length of the point cloud data in the X direction is set to Lx and the length in the Y direction is set to Ly, the number of columns Nx and Ny are expressed by equations (9) and (10), respectively.

[0261] Nx=(Lx / Rx)+1……(9)

[0262] Ny=(Ly / Ry)+1……(10)

[0263] Since the lengths Lx and Ly are 5mm and 45mm respectively, the number of point cloud columns Nx and Ny are 101 and 301 respectively. It should be noted that in equations (9) and (10), the second term on the right represents adding a column including the origin.

[0264] On the other hand, by replacing the X-direction resolution Rx of the shape data in equation (9) with the X-direction resolution Rx0 of the sample shape data, and replacing the Y-direction resolution Ry of the shape data in equation (10) with the Y-direction resolution Ry0 of the sample shape data, the number of point cloud columns Nx' and Ny' after reconstructing the shape data are obtained. Specifically, the number of point cloud columns Nx' and Ny' are 201 and 451, respectively. That is, the original shape data is reconstructed into point cloud data consisting of 201 point cloud columns in the X direction and 451 point cloud columns in the Y direction.

[0265] Furthermore, for each point in the reconstructed point cloud data, based on equation (4), the Z(Xn / Cx, Ym / Cy) coordinates shown in equation (4) are calculated respectively, and the Z coordinate values ​​are corrected, thus completing the resolution conversion correction. Then, in the first determination unit 28, the shape of the welded part 201 is determined to be good or bad using the learned determination model.

[0266] By correcting the resolution of the shape data to match that of the sample shape data, misidentification of shape defects in the judgment model can be suppressed when the shape data is input into the judgment model. Therefore, even when shape data is obtained under conditions different from those when the sample shape data was obtained, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0267] Example 2

[0268] Figure 13The diagram shows a top view of the welded portion according to Embodiments 2 and 3. In the top view, the welded portion 201 is approximately L-shaped with a corner. Furthermore, the welded portion 201 is divided into sections I and II, which are straight sections; section III, which is a corner approximately quarter-circle shaped; and sections IV and V, which are straight sections.

[0269] In this embodiment, the visual inspection of the range from interval I to interval III is described.

[0270] In section I, which is a straight section, it is difficult to produce small shape defects. Therefore, in the visual inspection of section I, compared with improving the measurement resolution to obtain the shape data of the welded part 201, it is preferable to reduce the time spent on visual inspection to reduce production cycle time.

[0271] On the other hand, in zone III, which is the corner area, the frequency of smaller shape defects tends to be higher than in zone I. Therefore, in the visual inspection of zone III, it is necessary to obtain the shape data of the welded part 201 by increasing the measurement resolution, thereby performing a high-precision visual inspection.

[0272] Therefore, in this embodiment, intervals I and III are set as constant-speed intervals, and the moving speed V1 of robot 16 in interval I is set to 9 m / min, and the moving speed V3 of robot 16 in interval III is set to 3 m / min. Additionally, interval II is set as a deceleration interval where the moving speed V decelerates from V1 to V3 as a linear function of time t. It should be noted that the speed control function V(t) of robot 16 in interval II has the same form as shown in equation (8).

[0273] The measurement frequency F and the X-direction resolution Rx in intervals I and III, which are constant velocity intervals, were set to the same value as in Example 1, 0.05 mm (=50 μm). Furthermore, the Y-direction resolution Ry in interval I was also set to the same value as in Example 1, 0.15 mm (=150 μm). On the other hand, as the moving speed V decreased from V1=9 m / min (interval I) to V3=3 m / min (interval III), the Y-direction resolution Ry in interval III decreased to 0.05 mm (=50 μm).

[0274] Furthermore, in each of intervals I and III, which are constant velocity intervals, the number of point cloud columns Nx and Ny when acquiring shape data are set to the same values ​​as in Example 1. That is, the number of point cloud columns Nx and Ny when acquiring shape data are 101 and 301, respectively.

[0275] On the other hand, in interval II, which is the deceleration interval, the X-direction resolution Rx, the number of point cloud columns Nx when acquiring shape data, and the measurement frequency F are set to the same values ​​as in interval I and interval III. That is, the X-direction resolution Rx is 0.05mm (=50μm), the number of point cloud columns Nx is 101, and the measurement frequency F is 1000Hz.

[0276] Additionally, the period T (refer to) will be used as the deceleration period in interval II. Figure 11 Let 0.2see be the constant. Therefore, in the speed control function V(t) shown in equation (8), when V1 = 9 m / min, V2 = (V3 = )3 m / min, and T = 0.2 sec are substituted respectively, the first coefficient A of time t in equation (8) is (V2 - V1) / T = -30. In addition, the constant B = (V1 = )9.

[0277] In this case, when the values ​​of equation (8) and the measured frequency F (Hz) are substituted into equation (5), the Y-direction resolution Ry(t) in interval II is expressed in the form shown in equation (11).

[0278] Ry(t)=(50 / 3F)×V(t)=(1 / 60)×(-30t+9)

[0279] =-0.5t+0.15……(11)

[0280] Furthermore, substituting the Y-direction resolution Ry(t) shown in equation (11) into equation (6) yields the resolution coefficient Cym(t). It should be noted that the travel time Tm(sec) of the travel time of interval II (object interval) in this case is expressed in the form shown in equation (12).

[0281] 0≤Tm=m / F=m / 1000≤T=0.2……(12)

[0282] Here, m represents the m-th position from the origin of the object interval.

[0283] In this case, the number of point cloud columns Nx' after reconstructing the shape data is 201. This is obtained by replacing the X-direction resolution Rx of the shape data in equation (9) with the X-direction resolution Rx0 of the sample shape data. Here, the X-direction resolution Rx0 (0.025 mm (=25 μm)) and the length Lx of the measurement range (scanning range) of the shape measurement unit 21 in the X direction are 5 mm. On the other hand, the number of point cloud columns Ny' after reconstructing the shape data is expressed in the form shown in equation (13).

[0284] [Mathematical Expression 2]

[0285]

[0286] As described above, the original shape data is reconstructed into point cloud data consisting of 201 point cloud columns in the X direction and 201 point cloud columns in the Y direction. For the XY coordinates (Xn, Ym), Z(Xn / Cx, Ym / Cy) is calculated as the corrected coordinates in the manner satisfying equation (7). m (t)).

[0287] In this embodiment, n is an integer greater than or equal to 1, satisfying 1 ≤ n ≤ 201, and m is an integer greater than or equal to 1, satisfying 1 ≤ m ≤ 201.

[0288] For all point clouds contained in interval II, calculate Z(Xn / Cx, Ym / Cy) as the corrected coordinates, as shown in equation (7). m (t)). Furthermore, for all coordinate points contained in interval II, calculate (Xn / Cx, Ym / Cy). m (t), Z(Xn / Cx, Ym / Cy) m (t))), in interval II, the resolution-corrected shape data is obtained through such resolution conversion correction.

[0289] In each of intervals I to III, after obtaining the resolution-corrected shape data through resolution conversion correction, the first determination unit 28 determines whether the shape of the welded part 201 is good or bad using the learned determination model.

[0290] By utilizing resolution conversion correction, the resolution of the shape data measured by the shape measurement unit 21 is corrected to a value equal to the resolution of the sample shape data. This allows for the suppression of misidentification of shape defects in the judgment model when the shape data is input. Here, the judgment model is generated in advance based on and further reinforced by multiple learning datasets, which are generated from the sample shape data. Therefore, even when shape data is obtained under conditions different from those when the sample shape data was obtained, the shape of the welded part 201 can be evaluated with high accuracy, and the goodness or badness of the shape can be correctly determined.

[0291] Furthermore, in the deceleration interval (interval II) where the resolution changes over time, by performing the aforementioned resolution conversion correction on the shape data, misidentification of shape defects in the judgment model can be suppressed when the shape data is input into the judgment model. Therefore, even in the deceleration interval, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0292] Example 3

[0293] In this embodiment, the following describes the application of... Figure 13 The visual inspection was carried out on the range of interval I to interval V.

[0294] In intervals I to III, the shape data resolution conversion correction was performed using the method shown in Example 2, and the welded part 201 was visually inspected. Additionally, interval V is a constant-speed interval with a moving speed of V = (V5 =) 12 m / min. That is, in interval V, the shape data resolution conversion correction was performed using the same method as in intervals I and III of Examples 1 and 2, and the welded part 201 was visually inspected. Therefore, the explanation here focuses on the resolution conversion correction in interval IV.

[0295] Interval IV is the acceleration interval in which the moving speed V accelerates from V3 to V5 as a linear function of time t. It should be noted that the speed control function V(t) of robot 16 in interval IV, which is the acceleration interval, has the same form as shown in equation (8).

[0296] In interval IV, the X-direction resolution Rx of the measured shape data, the number of point cloud columns Nx when the shape data was acquired, and the measurement frequency F are set to the same values ​​as in intervals I and III, respectively. That is, the X-direction resolution Rx is 0.05 mm (=50 μm), the number of point cloud columns Nx is 101, and the measurement frequency F is 1000 Hz.

[0297] Furthermore, the acceleration period T in interval IV is set to 0.2 sec. Therefore, in the speed control function V(t) shown in equation (8), when V1 = (V3 =) 3 m / min, V2 = (V5 =) 12 m / min, and T = 0.2 sec are substituted respectively, the first coefficient A of time t in equation (8) is (V2 - V1) / T = 45. In addition, the constant B = (V3 =) 3.

[0298] In this case, when the values ​​of equation (8) and the measured frequency F are substituted into equation (5), the Y-direction resolution Ry(t) in interval IV is expressed in the form shown in equation (14).

[0299] Ry(t)=(50 / 3F)×V(t)=(1 / 60)×(45t+3)

[0300] =0.75t+0.05……(14)

[0301] Furthermore, substituting the Y-direction resolution Ry(t) shown in equation (14) into equation (6) yields the resolution coefficient Cy. m(t). It should be noted that the travel time Tm in this case is also expressed in the form shown in equation (12).

[0302] In addition, in this case, the number of point cloud columns Nx' after reconstructing the shape data is also 201. On the other hand, the number of point cloud columns Ny' after reconstructing the shape data is expressed in the form shown in Equation (15).

[0303] [Mathematical Expression 3]

[0304]

[0305] As described above, the original shape data is reconstructed into point cloud data consisting of 201 point cloud columns in the X direction and 251 point cloud columns in the Y direction. For the XY coordinates (Xn, Ym), Z(Xn / Cx, Ym / Cy) is calculated in a manner that satisfies Equation (7). m (t)).

[0306] In this embodiment, n is an integer greater than or equal to 1, satisfying 1≤n≤201, and m is an integer greater than or equal to 1, satisfying 1≤m≤251.

[0307] For all point clouds contained in interval IV, calculate Z(Xn / Cx, Ym / Cy) as the corrected coordinates, as shown in equation (7). m (t)). Furthermore, for all coordinate points contained in interval IV, calculate (Xn / Cx, Ym / Cy). m (t), Z(Xn / Cx, Ym / Cy) m (t))), in interval IV, we obtain the shape data with the resolution corrected.

[0308] In each interval from interval I to interval V, after obtaining the shape data with the resolution corrected, the first determination unit 28 determines whether the shape of the welded part 201 is good or bad by using the learned determination model.

[0309] In this way, by converting and correcting the resolution of the shape data to make it equal to the resolution of the sample shape data, misidentification of shape defects in the judgment model can be suppressed when the shape data is input into the judgment model. Therefore, even when the shape data is obtained under different inspection conditions than when the sample shape data was obtained, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0310] Furthermore, in the deceleration interval (interval II) and acceleration interval (interval IV) where the resolution changes over time, the aforementioned resolution conversion correction is applied to the shape data. This allows for the suppression of misidentification of shape defects in the judgment model when the shape data is input into the model. Therefore, the shape of the welded part 201 can be evaluated with good accuracy in both the deceleration and acceleration intervals, and the goodness or badness of the shape can be correctly determined.

[0311] Example 4

[0312] As mentioned earlier, without performing resolution conversion correction, it is preferable to make the resolution of the shape data obtained by the shape measuring unit 21 equal to the resolution of the sample shape data. In this way, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0313] Furthermore, when acquiring shape data, it is preferable to keep the scanning speed of the shape measuring unit 21 in the Y direction, i.e., the moving speed V of the robot 16, constant throughout the inspection interval. In this way, the resolution of the shape data can be kept constant throughout the inspection interval.

[0314] Therefore, shape data is typically obtained using the following method. First, the shape measuring unit 21 is accelerated from a stationary state until it reaches the beginning of the inspection interval. In this case, the moving speed V reaches the desired value at the moment the shape measuring unit 21 reaches the beginning of the inspection interval. Then, the shape of the welded part 201 is measured while the shape measuring unit 21 moves at a constant speed V throughout the entire inspection interval. In this way, the resolution of the shape data remains constant throughout the entire inspection interval, enabling accurate evaluation of the shape of the welded part 201 and correct determination of its quality.

[0315] However, this method requires setting an area outside the inspection area, namely the assisting area. Even in the assisting area, the robot 16 needs to be moved, which increases the production cycle time required for appearance inspection.

[0316] On the other hand, according to the method disclosed in this application, even in the acceleration / deceleration range where the moving speed V changes, the resolution of the shape data can be converted and corrected to a value equal to the resolution of the sample shape data. Therefore, the shape of the welded portion 201 can be evaluated without setting an auxiliary range. The following description uses this embodiment.

[0317] Figure 14The illustration schematically depicts the visual inspection of the welded portion in Embodiment 4. For the straight weld seam, i.e., the welded portion 201, the shape measuring unit 21 is moved as shown below to perform the visual inspection. In this embodiment, starting from a stationary state, the shape measuring unit 21 is accelerated while the robot 16 is moved. In this case, the speed control function V(t) of the robot 16 is the same as that shown in Equation (8). Furthermore, from the moment the speed Vc (=9m / min) is reached, the shape measuring unit 21 is moved at a constant speed while maintaining this speed. Here, the shape measuring unit 21 continuously measures the shape of the welded portion 201 from the moment the shape measuring unit 21 begins to move, thus performing the visual inspection.

[0318] The interval from the moment the shape measuring unit 21 begins to move until it reaches speed Vc is the acceleration interval, which also corresponds to the aforementioned assist interval. Therefore, even without setting an assist interval outside the inspection interval as in conventional methods, the resolution of the shape data is converted and corrected during the acceleration and deceleration intervals where the moving speed V changes, making it equal to the resolution of the sample shape data. Thus, the resolution of the acquired shape data can be corrected to be equal to the resolution of the sample shape data throughout the entire inspection interval, including the acceleration interval.

[0319] exist Figure 13 In the acceleration interval shown, the X-direction resolution Rx of the measured shape data, the number of point cloud columns Nx when the shape data is acquired, and the measurement frequency F are set to the same as those of the previous values. Figure 13 The values ​​shown are the same for the constant velocity intervals. That is, the X-direction resolution Rx is 0.05 mm (=50 μm), the number of point cloud columns Nx is 101, and the measurement frequency F is 1000 Hz. It should be noted that the number of point cloud columns Ny for the shape data in the Y-direction within the constant velocity interval is set to 301.

[0320] Furthermore, the acceleration period T in the acceleration interval is set to 0.3 sec. Therefore, in the speed control function V(t) shown in equation (8), when V1 = 0 m / min, V2 = (Vc = )9 m / min, and T = 0.3 sec respectively, the first-order coefficient A of time t in equation (8) is (V2 - V1) / T = 30. In addition, the constant B = 0, and the speed control function V(t) = 30t.

[0321] In this case, when the values ​​of equation (8) and the measured frequency F (Hz) are substituted into equation (5), the Y-direction resolution Ry(t) in the acceleration interval is expressed in the form shown in equation (14).

[0322] Ry(t)=(50 / 3F)×V(t)=(1 / 60)×(30t)

[0323] =0.5t……(16)

[0324] Furthermore, substituting the Y-direction resolution Ry(t) shown in equation (16) into equation (6) yields the resolution coefficient Cy. m (t). It should be noted that the travel time Tm (sec) in this case is also expressed in the form shown in equation (12).

[0325] Furthermore, in this case, similarly to interval II during the deceleration period in Example 2 and interval IV during the acceleration period in Example 3, the number of point cloud columns Nx' after reconstructing the shape data is 201. On the other hand, the number of point cloud columns Ny' after reconstructing the shape data is expressed in the form shown in Equation (17).

[0326] [Mathematical Expression 4]

[0327]

[0328] As described above, the original shape data is reconstructed into point cloud data consisting of 201 point cloud columns in the X direction and 226 point cloud columns in the Y direction. For the XY coordinates (Xn, Ym), Z(Xn / Cx, Ym / Cy) is calculated as the corrected coordinates in the manner satisfying equation (7). m (t)).

[0329] In this embodiment, n is an integer greater than or equal to 1, satisfying 1≤n≤201, and m is an integer greater than or equal to 1, satisfying 1≤m≤226.

[0330] For all point clouds contained in the acceleration interval, calculate Z(Xn / Cx, Ym / Cy) as the corrected coordinates, as shown in equation (7). m (t)). Furthermore, for all coordinate points contained within the acceleration interval, calculate (Xn / Cx, Ym / Cy). m (t), Z(Xn / Cx, Ym / Cy) m (t))), in the acceleration interval, the resolution of the shape data is transformed to obtain shape data with the resolution corrected.

[0331] In each interval of the entire inspection interval, namely the acceleration interval and the constant speed interval, after obtaining the shape data with the resolution corrected, the first determination unit 28 determines whether the shape of the welded part 201 is good or bad by using the learned determination model.

[0332] In this way, by converting the resolution of the shape data, the resolution of the shape data measured by the shape measurement unit 21 is corrected to a value equal to the resolution of the sample shape data. Therefore, when the shape data is input into the judgment model, misidentification of shape defects in the judgment model can be suppressed. Here, the judgment model is generated in advance based on each of multiple learning datasets, which are generated based on the sample shape data. Therefore, even when shape data is obtained under conditions different from the inspection conditions when the sample shape data was obtained, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0333] Furthermore, during the acceleration and deceleration intervals where the moving speed V changes, the resolution of the shape data is adjusted by converting it to a value equal to that of the sample shape data. Thus, a portion of the inspection interval for shape data measurement is used as an acceleration interval, eliminating the need for an auxiliary interval outside the inspection interval. This allows for visual inspection of the welded portion 201 from a stationary state of the shape measuring unit 21. Since no auxiliary interval is required, inspection time is shortened, thereby reducing production cycle time, including visual inspection.

[0334] Furthermore, even in the accelerated inspection period where the resolution changes over time, by performing the aforementioned resolution conversion correction on the shape data, misidentification of shape defects in the judgment model can be suppressed when the shape data is input into the judgment model. Therefore, the shape of the welded part 201 can be evaluated with good accuracy throughout the entire inspection period, including the accelerated inspection period, and the goodness or badness of the shape can be correctly determined.

[0335] Example 5

[0336] Figure 15 This is a schematic diagram showing the Z-direction profile of the welded area involved in Embodiment 5. Figure 16A This is a schematic diagram showing the arrangement of the shape measuring units in interval A. Figure 16B This is a schematic diagram showing the arrangement of the shape measuring units in interval B. Figure 16C This is a schematic diagram showing the arrangement of the shape measuring units in interval C. Figure 15 The welded part 201 shown has a section A as a flat part, a section B as a forward-leaning part as viewed from the moving direction, and a section C as a backward-leaning part as viewed from the moving direction in the inspection interval. The sections A, B, and C repeat periodically in the moving direction.

[0337] Figure 17A This is a schematic diagram showing the arrangement of the shape measuring unit involved in the conventional method, that is, the arrangement when checking interval A. Figure 17B This is a schematic diagram showing the arrangement of the shape measuring unit involved in the conventional method, that is, the arrangement when checking interval B.

[0338] The shape of the welded part 201 varies, for example, Figure 15 As shown, there is also a welding part 201 with a protrusion that protrudes in the Z direction periodically.

[0339] In the Figure 15 When visually inspecting the welded portion 201 shown, the inspection area includes three sections: section A, which is a flat portion; section B, which is a forward-leaning portion as observed from the movement direction of the robot 16; and section C, which is a backward-leaning portion as observed from the movement direction of the robot 16. It should be noted that the height of the protrusion in the Z direction, based on the surface of the workpiece 200, is H1.

[0340] When using the aforementioned three-dimensional shape measuring sensor as the shape measuring unit 21, the appearance inspection of the welded part 201 is performed while keeping the distance between the light-receiving surface of the camera or light-receiving sensor matrix 21b and the surface of the welded part 201, i.e., the working distance, constant. This is because if the working distance changes, the shape cannot be accurately measured. Therefore, in the inspection area, in cases where the height or inclination of the surface of the welded part 201 changes significantly, it is necessary to change the arrangement of the shape measuring unit 21 before and after that area to maintain the working distance. Specifically, it is necessary to change the angle of the light-receiving surface of the shape measuring unit 21 relative to the direction of movement.

[0341] In the Figure 15 When visually inspecting the welded part 201 shown, in section A, as follows: Figure 16A As shown, the shape measuring unit 21 is arranged such that the optical axis of the laser beam received by the shape measuring unit 21 is perpendicular to the surface of the welding part 201, which in this case is the surface of the workpiece 200. In interval B, as... Figure 16B As shown, the shape measuring unit 21 is arranged such that a forward angle θ1 is formed according to the inclination of the protrusion, so that the optical axis of the laser beam received by the shape measuring unit 21 is perpendicular to the inclined surface of the forward-tilting portion, i.e., interval B, as observed from the moving direction. The forward angle θ1 is the angle at which the shape measuring unit 21 tilts in the direction opposite to the moving direction, relative to the vertical direction when the surface of the workpiece 200, which is the flat portion of interval A, is taken as a reference. In interval C, as... Figure 16CAs shown, the shape measuring unit 21 is arranged such that a back angle θ2 is formed according to the degree of inclination of the protrusion, so that the optical axis of the laser beam received by the shape measuring unit 21 is perpendicular to the inclined surface of the inclined portion, i.e., the interval C, as observed from the moving direction. The back angle θ2 is the angle at which the shape measuring unit 21 is inclined towards the moving direction relative to the vertical direction when the surface of the workpiece 200, which is the interval A, is taken as the reference.

[0342] On the other hand, when visual inspection is performed continuously while the arrangement of the shape measuring unit 21 is changed, the working distance will also change before and after the changed part, so the shape of the welded part 201 cannot be accurately evaluated.

[0343] Therefore, in conventional methods, the shape measuring unit 21 is temporarily stopped before and after the location where the height or inclination of the surface of the welding part 201 changes significantly. After the arrangement is changed, the shape measuring unit 21 is moved again, and the shape measurement of the welding part 201 is restarted. In this case, in conventional methods, as described in Embodiment 4, an auxiliary range needs to be provided outside the inspection range for measuring shape data. For example, as... Figure 17A As shown, an auxiliary zone of a specified distance needs to be set behind (in the opposite direction of movement) the starting position of the inspection zone, i.e., the measurement object zone (the aforementioned zone A, which is the flat section). Additionally, as... Figure 17B As shown, an auxiliary zone of a specified distance needs to be set behind (in the opposite direction of movement) the starting position of the inspection zone, i.e., the measurement object zone (the aforementioned zone B, which is the forward tilt portion observed from the direction of movement). It should be noted that... Figure 17B In the case shown, since the assist range also includes a component along the Z direction, it is necessary to move the shape measuring unit 21 along the Z direction as well.

[0344] However, in the shape measurement of the welding section 201, if an auxiliary zone is set, there is a technical problem of increased production cycle time, which is the same as the situation described above. In particular, if... Figure 15 As shown, when the welding section 201 includes multiple sections with significant variations in the height or inclination of the surfaces, the increase in production cycle time is even more pronounced.

[0345] On the other hand, according to the method disclosed in this application specification, as described above, the shape of the welded portion 201 can be evaluated without setting up an auxiliary zone outside the inspection zone. Therefore, the increase in production cycle time can be significantly suppressed. Hereinafter, this embodiment will be used for description.

[0346] In measurement Figure 15When welding position 201 is shown, the scanning distances La to Lc of the shape measuring unit 21 in each interval A to C are essentially equal, all being 67.5 mm. In this case, the scanning distance refers to the distance traveled by the shape measuring unit 21 along the surface of the welding position 201.

[0347] Preliminary experiments show that it takes 0.4 seconds to accelerate the shape measuring unit 21 from a stationary state to a certain speed of 9 m / min. In addition, the moving distance of the shape measuring unit 21 during acceleration is 30 mm. Therefore, in each interval from interval A to interval C, the scanning distance in the acceleration interval is 30 mm, and the scanning distance in the constant speed interval is 37.5 mm.

[0348] In addition, from Figure 15 It can be seen that the welding part 201 includes six intervals A, five intervals B, and five intervals C. Therefore, in the intervals (6+5+5=16) that are the objects of shape measurement, the total scanning distance of the acceleration interval is 30 (mm) × 16 (intervals) = 480mm, and the total scanning distance of the constant speed interval is 37.5 (mm) × 16 (intervals) = 600mm.

[0349] Furthermore, in the scanning time (equivalent to the movement time of robot 16) of the shape measuring unit 21 scanning the welding part 201, the scanning time in the acceleration zone is 16 (locations) × 0.4 (sec) = 6.4 (sec). The scanning time in the constant velocity zone is 16 (locations) × 37.5 (sec) / 9 (m / min) = 16 × 37.5 (sec) / 150 (mm / sec) = 4.0 (sec). These are summarized in Table 1.

[0350] [Table 1]

[0351]

[0352] It should be noted that the welded part 201 was visually inspected using the same method as in Example 4. That is, in each constant speed interval, the original shape data was reconstructed using the above-mentioned X-direction resolutions Rx, Rx0 and Y-direction resolutions Ry, Ry0. For each point in the reconstructed point cloud data, the Z-coordinate value, which is the coordinate after correction processing, was corrected based on Equation (4), thereby correcting the resolution of the shape data. Furthermore, in the first determination unit 28, the goodness or badness of the shape of the welded part 201 was determined in each constant speed interval by using a determination model that was generated based on each learning dataset in the multiple learning datasets and further reinforced by the learning model. It should be noted that, as described above, each learning dataset in the multiple learning datasets was generated based on the sample shape data obtained in advance.

[0353] In addition, within each acceleration interval, the resolutions Rx and Rx0 in the X direction and Ry(t) and Ry0 in the Y direction are used, along with the resolution coefficient Cy. m (t), reconstruct the original shape data, and for all point clouds contained in the acceleration interval, calculate Z(Xn / Cx, Ym / Cy) as the corrected coordinates shown in equation (7). m (t)). Furthermore, for all coordinate points contained within the acceleration interval, calculate (Xn / Cx, Ym / Cy). m (t), Z(Xn / Cx, Ym / Cy) m In each acceleration interval, shape data with corrected resolution is obtained. Then, in the first determination unit 28, the shape of the welded part 201 is determined to be good or bad in each acceleration interval by using the learned determination model.

[0354] On the other hand, a comparative example is taken where the shape of the welded portion 201 is measured using a conventional method. In this comparative example, as mentioned earlier, when shape measurement is performed in each of intervals A to C, an auxiliary interval is provided in addition to the inspection interval where shape data is measured. Furthermore, in this comparative example, at the end point of measurement in interval A, the shape measuring unit 21 is moved back a distance corresponding to the auxiliary interval. Then, the angle between the light-receiving surface of the shape measuring unit 21 and the surface of the welded portion 201 is changed so that the optical axis of the laser beam received by the shape measuring unit 21 is perpendicular to the surface of the workpiece 200. Then, the shape measuring unit 21 is moved to the measurement start point while accelerating. Finally, the shape measuring unit 21 measures the shape of the welded portion 201 while moving at a constant speed from the measurement start point to the measurement end point in interval A. Such steps were performed in each of the six intervals A (flat sections), the five intervals B (forward tilt sections as observed from the direction of movement), and the five intervals C (backward tilt sections as observed from the direction of movement) to obtain the shape data of the welded part 201.

[0355] Furthermore, with the robot 16's moving speed V fixed at Vc (=9m / min), sample shape data was acquired, and a judgment model, enhanced through learning based on this sample shape data, was prepared in advance. Using this judgment model, the shape of the welded part 201 was evaluated. Then, in the first judgment unit 28, the learned judgment model was used to determine the quality of the welded part 201's shape in each interval from interval A to interval C. These results were then summarized across the entire inspection interval, and the final judgment result was output.

[0356] In addition, in the comparative example, the scanning distance and scanning time of the shape measuring unit 21 in each auxiliary interval set in addition to the inspection interval where shape data is measured are added to the scanning distance and scanning time of the constant speed interval to calculate the total interval of scanning distance and scanning time (auxiliary interval + constant speed interval). Table 2 shows the summary results including the comparison with the results of this embodiment. It should be noted that in the auxiliary interval, the shape measuring unit 21 is moved back and forth with the measurement start point of each interval as the base point. Therefore, the scanning distance and scanning time of the shape measuring unit 21 in the auxiliary interval are twice the values ​​of this embodiment. That is, the scanning distance of the auxiliary interval is 30 (mm) × 2 = 60 (mm) per interval, and the scanning time is 0.4 (sec) × 2 = 0.8 (sec) per interval.

[0357] [Table 2]

[0358]

[0359] As shown in Table 2, compared to the comparative example, in this embodiment, the scanning distance of the shape measuring unit 21 can be reduced by about 30%, and the scanning time can be reduced by about 50%. That is, the increase in production cycle time required for appearance inspection can be significantly suppressed.

[0360] Furthermore, by correcting the resolution of the shape data measured by the shape measuring unit 21 to a value equal to the resolution of the pre-obtained sample shape data, misidentification of shape defects in the judgment model can be suppressed when the shape data is input into the judgment model. Therefore, even when shape data is obtained under conditions different from those when the sample shape data was obtained, the shape of the welded part 201 can be evaluated with good accuracy, and the goodness or badness of the shape can be correctly determined.

[0361] In addition, such as Figure 15 As shown, even when the height or inclination of the welded part 201 varies greatly, the welded part 201 can be visually inspected without setting up an auxiliary zone outside the inspection zone by dividing each zone into an acceleration zone and a constant speed zone in the inspection zone for measuring shape data.

[0362] Furthermore, during the acceleration interval where the resolution changes over time, the shape data undergoes the aforementioned resolution conversion correction to make it equal to the resolution of the pre-obtained sample shape data. This allows for the suppression of misidentifications of shape defects in the judgment model, which is generated from multiple learning datasets and further reinforced through training, when the shape data is input into the judgment model. It should be noted that multiple learning datasets are generated in advance based on the sample shape data. Therefore, the shape of the welded part 201 can be evaluated with good accuracy throughout the entire inspection interval, including the acceleration interval, and the goodness or badness of the shape can be correctly determined.

[0363] (Other implementation methods)

[0364] exist Figure 1 The example shown depicts a robot 16 equipped with both a welding torch 11 (welding head 11) and a shape measuring unit 21. However, it is also possible to install another robot (not shown) equipped with the shape measuring unit 21, in addition to the robot 16 equipped with the welding torch 11 (welding head 11). In this case, various data are sent from the control unit (not shown) of another robot that controls the movement of the other robot to the data processing unit 23.

[0365] In addition, the learning dataset generation unit 26 shown in the embodiment classifies the sample shape data according to each material and shape of the workpiece 200, and performs data augmentation processing on the classified sample shape data to generate multiple learning datasets.

[0366] However, the learning dataset generation unit 26 may also lack this classification function. In this case, the decision model generation unit 27 may also lack the function of generating decision models according to each material and shape of the workpiece 200.

[0367] -Industry Applicability-

[0368] The visual inspection apparatus disclosed herein can accurately evaluate the three-dimensional shape of the welded parts even when the inspection conditions change, and is therefore particularly useful in the visual inspection of workpieces containing various welded parts.

[0369] -Symbol Explanation-

[0370] 10 Welding Equipment

[0371] 11. Welding head (welding torch)

[0372] 12 Welding wire

[0373] 13 Welding wire feeding device

[0374] 14 Power Supply

[0375] 15 Output Control Unit

[0376] 16 robots

[0377] 17. Robot Control Department

[0378] 20. Visual inspection device

[0379] 21 Shape Measurement Department

[0380] 22 Sensor Control Unit

[0381] 23 Data Processing Department

[0382] 24 Shape Data Processing Department

[0383] 25 First Storage Division

[0384] 26 Learning Dataset Generation Department

[0385] 27. Judgment Model Generation Department

[0386] 28 First Judgment Department

[0387] 29 Notification Department

[0388] 100 Welding System

[0389] 200 workpieces

[0390] 201 Welding area.

Claims

1. A visual inspection device for inspecting the appearance of welded parts of a workpiece, characterized in that: The appearance inspection device includes at least a shape measuring unit and a data processing unit. The shape measuring unit is mounted on the robot and measures the three-dimensional shape of the welded part along the welding line. The data processing unit processes the shape data obtained by the shape measurement unit. The data processing unit includes at least a shape data processing unit, a learning dataset generation unit, a decision model generation unit, and a first decision unit. The shape data processing unit performs at least resolution correction processing on the shape data obtained by the shape measurement unit. The learning dataset generation unit performs data augmentation processing on multiple sample shape data pre-obtained by the shape measurement unit to generate multiple learning datasets. The judgment model generation unit uses multiple training datasets to generate a judgment model for determining whether the shape of the welded part is good or bad. The first determination unit determines whether the shape of the welded part is good or bad based on the shape data corrected by the shape data processing unit and one or more determination models generated by the determination model generation unit. Based on the robot's moving speed, the data processing unit divides the scanning range of the shape measuring unit into a constant speed range and an acceleration / deceleration range. The shape data processing unit corrects the resolution of the shape data in the selected interval from the intervals segmented by the data processing unit.

2. The appearance inspection device according to claim 1, characterized in that: The shape data processing unit corrects the resolution of the shape data obtained by the shape measuring unit based on the measuring resolution, measuring frequency, and scanning speed of the shape measuring unit.

3. The appearance inspection device according to claim 2, characterized in that: The sample shape data is obtained under the conditions of pre-setting the measurement resolution, measurement frequency, and scanning speed. The shape data processing unit corrects the resolution of the shape data obtained by the shape measurement unit, so that the resolution of the shape data obtained by the shape measurement unit becomes a value equal to the resolution of the sample shape data.

4. The appearance inspection device according to claim 2 or 3, characterized in that: The appearance inspection device further includes a sensor control unit, which stores the inspection conditions of the shape measuring unit and sends the stored inspection conditions to the data processing unit. The sensor control unit sends the measurement resolution and measurement frequency in the X direction to the data processing unit. The X direction intersects the Y direction (the direction extending along the welding line) and the Z direction (the height direction of the welding part), respectively. The data processing unit receives the robot's movement speed or the robot's speed control function from the robot control unit that controls the robot's movements.

5. The appearance inspection device according to claim 4, characterized in that: The shape data processing unit corrects the Z-direction value of the shape data based on the X-direction resolution and Y-direction resolution of the shape data. While the robot moves at a constant speed along the welding line, the shape measuring unit measures the three-dimensional shape of the welding area. The Y-direction resolution is determined based on the measurement frequency and the robot's moving speed. When the robot accelerates and / or decelerates within a predetermined interval extending along the welding line, and the shape measuring unit measures the three-dimensional shape of the welded area, The Y-direction resolution is determined based on the measurement frequency and the robot's speed control function. The robot's speed control function is a time-dependent function.

6. The appearance inspection device according to any one of claims 1 to 3, characterized in that: The learning dataset generation unit classifies the multiple sample shape data obtained by the shape measurement unit according to each material and shape of the workpiece, and performs data augmentation processing on the classified sample shape data to generate multiple learning datasets. The determination model generation unit uses multiple learning datasets to generate the determination model according to each material and shape of the workpiece.

7. The appearance inspection device according to any one of claims 1 to 3, characterized in that: The data processing unit further includes a first storage unit, which stores at least the sample shape data. The sample shape data stored in the first storage unit is read out to the learning dataset generation unit to generate multiple learning datasets.

8. The appearance inspection device according to any one of claims 1 to 3, characterized in that: The data processing unit further includes a notification unit, which notifies the first determination unit of the determination result.

9. The appearance inspection device according to any one of claims 1 to 3, characterized in that: The decision model is pre-trained and strengthened using the training dataset. The learning dataset contains qualified product data and learning data. The qualified product data refers to shape data that does not include shape defects at the welded area. The learning data refers to shape data (i.e., non-conforming product data) that contains shape defects at the welded location, which determines the type of shape defect and marks that type. The first determination unit inputs the shape data input from the shape data processing unit into the determination model. The determination model determines whether the shape defect exists, and determines the type, number, size, and position of the shape defect relative to the welded part. Based on the determination results, it determines whether the shape of the welded part is good or bad.

10. A welding system, characterized in that: include: The appearance inspection device according to any one of claims 1 to 9; as well as A welding apparatus for welding the workpiece. The welding device includes at least a welding head and an output control unit. The welding head is used to input heat to the workpiece. The output control unit controls the welding output of the welding head.

11. The welding system according to claim 10, characterized in that: The welding apparatus also includes the robot and the robot control unit. The robot holds the welding head and moves it to the desired position. The robot control unit controls the robot's movements. If the first determination unit determines that the shape of the welded part is defective, the output control unit stops the welding output of the welding head, and the robot control unit stops the robot's operation or makes the robot work by bringing the welding head to a predetermined initial position.

12. A method for correcting shape data, wherein the shape data is obtained by the appearance inspection device according to any one of claims 1 to 9, characterized in that: include: The shape measuring unit measures the three-dimensional shape of the welding part while moving with the robot, and obtains the sample shape data used to generate multiple learning datasets; The step of the shape measuring unit measuring the three-dimensional shape of the welding part while moving together with the robot, and obtaining the shape data; as well as When the resolution of the shape data obtained by the shape measurement unit is different from the resolution of the sample shape data. The step of the shape data processing unit correcting the shape data so that the resolution of the shape data obtained by the shape measurement unit becomes a value equal to the resolution of the sample shape data.

13. The method for correcting shape data according to claim 12, characterized in that: The shape data processing unit corrects the height value of the welding portion in the Z direction (i.e., the Z-direction) based on the X-direction resolution and Y-direction resolution of the shape data. While the robot moves at a constant speed along the welding line, the shape measuring unit measures the three-dimensional shape of the welding area. The Y-direction resolution is determined based on the robot's moving speed and the measurement frequency of the shape measuring unit. When the robot accelerates and / or decelerates within a predetermined interval extending along the welding line, and the shape measuring unit measures the three-dimensional shape of the welded area, The Y-direction resolution is determined based on the measurement frequency and the robot's speed control function. The robot's speed control function is a time-dependent function.

14. A method for visual inspection of welded parts, comprising using the visual inspection apparatus described in any one of claims 1 to 9, characterized in that: The visual inspection methods for the welded parts include at least the following: The shape measuring unit measures the three-dimensional shape of the welding part while moving with the robot, and obtains the sample shape data used to generate multiple learning datasets; The step of the determination model generation unit using multiple learning datasets to generate one or more determination models for determining whether the shape of the welded part is good or bad; The step of the shape measuring unit measuring the three-dimensional shape of the welding part while moving together with the robot, and obtaining the shape data; When the resolution of the shape data obtained by the shape measurement unit differs from the resolution of the sample shape data, the shape data processing unit corrects the shape data to make the resolution of the shape data obtained by the shape measurement unit equal to the resolution of the sample shape data; and The first determination unit determines whether the shape of the welded part is good or bad based on the shape data whose resolution has been corrected by the shape data processing unit and one or more determination models generated by the determination model generation unit.

15. The method for visual inspection of welded parts according to claim 14, characterized in that: The shape data processing unit corrects the height value of the welding portion in the Z direction (i.e., the Z-direction) based on the X-direction resolution and Y-direction resolution of the shape data. While the robot moves at a constant speed along the welding line, the shape measuring unit measures the three-dimensional shape of the welding area. The Y-direction resolution is determined based on the robot's moving speed and the measurement frequency of the shape measuring unit. When the robot accelerates and / or decelerates within a predetermined interval extending along the welding line, and the shape measuring unit measures the three-dimensional shape of the welded area, The Y-direction resolution is determined based on the measurement frequency and the robot's speed control function. The robot's speed control function is a time-dependent function.

16. The method for visual inspection of welded parts according to claim 14 or 15, characterized in that: The decision model is pre-trained and strengthened using the training dataset. The learning dataset contains qualified product data and learning data. The qualified product data refers to shape data that does not include shape defects at the welded area. The learning data refers to shape data (i.e., non-conforming product data) that contains shape defects at the welded location, which determines the type of shape defect and marks that type. The first determination unit determines whether the shape of the welded portion is good or bad by including: The sub-step of determining whether or not there is a shape defect is based on the shape data input from the shape data processing unit and the determination model; The sub-step of determining the number, size, and position of the shape defects relative to the welded area; as well as The sub-step of determining the type of shape defect. The first determination unit determines whether the shape of the welded part is good or bad based on the determination results and confirmation results of each sub-step.

Images