Weights file generation method, appearance inspection method, and appearance inspection device

Abstract

In this weights file generation method, non-defective product data and defective product data are respectively acquired. Contour data relating to a welded part (201) is acquired from the non-defective product data. The number, size, position, and type of shape defect of shape-defective parts (210) in the defective product data are identified, and the identification results are acquired as annotation data for each type of shape defect. The contour data and the annotation data are combined to generate sub-shape data for each type of shape defect. For each type of sub-shape data, n (n is the number of types of shape defects included in a region (202)) types of weights file are established. When the reliability of determining one type of shape defect using one type of weights file falls below a predetermined value, said one type of weights file is subjected to additional training.

Description

Weight file generation method, visual inspection method, and visual inspection device

[0001] The present disclosure relates to a weight file generation method, an appearance inspection method, and an appearance inspection device for determining whether the shape of a welded portion formed on a workpiece to be processed is acceptable or not.

[0002] Conventionally, when performing visual inspection of an item or a processed portion of an item, a visual inspection device has been disclosed that performs visual inspection of the object from an inspection image captured of the object based on a learning model obtained by machine learning using learning images (see, for example, Patent Document 1).

[0003] Furthermore, in order to improve the accuracy of the learning model, a configuration has been proposed in which a first learning model that re-evaluates the inspection results of an inspection device and a second learning model that re-evaluates the judgment results of the first learning model (see Patent Document 2). The method disclosed in Patent Document 2 executes the steps of having the first learning model judge defect candidates detected by a defect device, having the second learning model re-evaluate the judgment results of the first learning model, and having the first learning model re-evaluate the re-evaluation results of the second learning model. In this way, it is possible to have the first learning model learn a large amount of high-quality learning data, and the accuracy of the first learning model can be efficiently improved.

[0004] Patent No. 2020-035095 Patent No. 2022-170299

[0005] Visual inspection of welds formed on workpieces, which are objects to be welded, is becoming increasingly common using learning models that have been reinforced through machine learning, such as those disclosed in Patent Documents 1 and 2.

[0006] The learning model judges the shape quality of a specified area containing a weld based on the input shape data and outputs the judgment result. In this case, the shape data of the weld is acquired, and if there is a shape defect, the type, position relative to the weld, and size are labeled. Multiple pieces of shape data with such labels are prepared and input into the learning model as learning data to strengthen the learning of the learning model.

[0007] However, retraining a model that has already been created can result in a decrease in the detection rate for shape defects, or in other words, a decrease in the confidence value of the judgment results.

[0008] One possible reason for this is the wide variety of geometric defects in welding. In other words, multiple types of geometric defects may be formed in a weld, and the above-mentioned labeling is performed for each of these multiple types of geometric defects. In this case, if training data containing newly labeled geometric defects of some types is input into the learning model and re-learning is performed, the learning results may affect the inspection results for other types of geometric defects, which may result in a decrease in the reliability of the judgment results.

[0009] Furthermore, if the same learning model is used to continuously judge the appearance of welds, shape defects that were previously identifiable may no longer be identifiable, or the reliability of the judgment may decrease.

[0010] The present disclosure has been made in consideration of these points, and its purpose is to provide a weight file generation method, an appearance inspection method, and an appearance inspection device that can improve the reliability of judgment results regarding the shape quality of a specified area including a weld.

[0011] In order to achieve the above object, a weight file generation method according to the present disclosure is a weight file generation method that, when shape data that is three-dimensional shape data of a predetermined area including a welding point is input, outputs at least the presence or absence of a specific type of shape defect in the shape data, and includes a first step of acquiring good product data that is the shape data related to good products that do not include the shape defect in the area, and defective product data that is the shape data related to defective products that include n types of shape defects in the area (n is an integer of 2 or more and is the number of types of shape defects included in the area); a third step of identifying the number, size, and position of shape-defective locations in the defective product data and the type of shape defect at the shape-defective locations, and acquiring the identification results for each type of shape defect as annotation data; a fourth step of combining the contour data and the annotation data to generate sub-shape data for each type of shape defect; a fifth step of determining the n types of weight files for each type of sub-shape data; and a sixth step of additionally learning the one type of weight file.

[0012] The visual inspection method disclosed herein is a method for visual inspection of a workpiece using the weight file generated by the weight file generation method, and is characterized by comprising at least a 16th step of acquiring the shape data, a 17th step of inputting the shape data into the weight file, an 18th step of determining whether or not there is a shape defect in the area using the weight file, a 19th step of identifying the number, size, and position of the shape defect points in the area, and a 20th step of determining whether the shape of the area is good or bad based on the determination results and identification results using the weight file.

[0013] The visual inspection device according to the present disclosure comprises at least a shape measurement unit and a judgment unit, wherein the shape measurement unit measures the three-dimensional shape of the area including the welded area, and the judgment unit is set with n types (n is an integer of 2 or more) of weight files for judging shape defects of the three-dimensional shape included in a predetermined area including the welded area, and the judgment unit judges whether the shape of the area is good or bad by inputting the shape data into each of the n types of weight files, wherein the n types of weight files are provided for each type of shape defect, and the n types of weight files are set in a state where one type of weight file can be additionally learned.

[0014] According to the present disclosure, it is possible to improve the reliability of the judgment results regarding the shape acceptability of a predetermined area including a welded portion using a weight file. Furthermore, because the additional learning improves the reliability of the judgment results only for the selected weight file, it is possible to maintain the reliability of the judgment results of other weight files that are not subjected to additional learning without fluctuation.

[0015] 4B is a schematic diagram of a welding system according to the first embodiment; FIG. 4C is an example of a file structure of a judgment model; FIG. 4D is another example of a file structure of a judgment model; FIG. 4E is a schematic plan view of shape data of a region including a welded portion; FIG. 4F is a schematic plan view of the shape data shown in FIG. 3 after the contour of the welded portion has been defined; FIG. 4G is a schematic plan view of the shape data shown in FIG. 4A after annotations have been added; FIG. 4H is an enlarged view of a portion where spatter is present in FIG. 4B; FIG. 4H is an enlarged view of a portion where holes are present in FIG. 4B; FIG. 4G is an enlarged view of a portion where multiple types of shape defects are present in FIG. 4B; FIG. 4H is a flowchart showing a procedure for generating a weighting file according to the first embodiment; FIG. 4D is a schematic plan view of shape data when contour data of a welded portion is added; FIG. 4F is a flowchart showing a procedure for determining whether the shape of a region including a welded portion is acceptable; FIG. 4G is an example of an output image of the determination result of whether the shape of a region is acceptable; FIG. 4H is a flowchart showing a procedure for updating a weighting file according to the first embodiment; FIG. 4H is a schematic plan view of shape data when annotations related to multiple types of shape defects are simultaneously added; FIG. 4H is a schematic plan view of shape data when annotations related only to spatter are added; FIG. 4H is a schematic plan view of shape data when annotations related only to holes are added. FIG. 1 is a schematic plan view of shape data when annotations related to only pits are added. FIG. 2 is a schematic plan view of shape data when annotations related to multiple types of shape defects are simultaneously added and corrected. FIG. 3 is a schematic plan view of shape data when annotations related to only undetected spatter are added. FIG. 4 is a schematic plan view of shape data when annotations related to only overdetected spatter are corrected. FIG. 5 is a diagram showing the reliability of shape defect judgment when weight files generated by a conventional method and the method shown in embodiment 1 are used. FIG. 6 is a flowchart showing a procedure for generating a weight file according to embodiment 2. FIG. 7 is a flowchart showing an i-th subroutine. FIG. 8 is a flowchart showing a procedure for judging the shape acceptability of a region including a welded portion according to embodiment 2. FIG. 9 is a flowchart showing a procedure for generating a weight file according to embodiment 3. FIG. 10 is a schematic plan view of an example of data extension processing. FIG. 11 is a flowchart showing a procedure for generating a weight file according to embodiment 4. FIG. 12 is a flowchart showing an i-th subroutine.

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present disclosure, its applications, or its uses.

[0017] (Embodiment 1) [1: Configuration of Welding System] Fig. 1 is a schematic diagram of a welding system according to embodiment 1. Fig. 2A is an example of a file structure of a judgment model. Fig. 2B is another example of a file structure of a judgment model. In this specification, "orthogonal," "parallel," or "same" means orthogonal, parallel, or the same, including manufacturing tolerances and assembly tolerances of the components constituting welding system 100 and processing tolerances of workpiece 200. It does not mean that the objects to be compared are orthogonal, parallel, or the same in the strict sense.

[0018] As shown in FIG. 1 , a welding system 100 includes a judgment model generating device 10 , a visual inspection device 20 , and a welding device 30 .

[0019] [1-1: Configuration of the Decision Model Generating Device] The decision model generating device 10 is configured with one or more computers. Furthermore, the decision model generating device 10, excluding the first storage unit 11, the first input device 18, and the first display device 19, is configured with multiple processors. However, this is not particularly limited, and for example, the decision model generating device 10 may include a dedicated LSI with a communication function. As will be described later, the processor includes a GPU (Graphics Processing Unit) and a CPU (Central Processing Unit).

[0020] The determination model generating device 10 includes a first storage unit 11, a first segmentation executing unit 12, a weight file generating unit 13, a first determination unit 14, a first data control unit 15, a first data receiving unit 16, and a first data transmitting unit 17. The determination model generating device 10 also includes a first input device 18 and a first display device 19.

[0021] The first storage unit 11 stores shape data that has been acquired in advance. This shape data is usually transmitted from the appearance inspection device 20. However, separately acquired shape data may also be stored.

[0022] Here, "shape data" refers to point cloud data that represents the three-dimensional shape of welded portion 201 (see FIG. 4A) formed on workpiece 200 and region 202 (see FIG. 4A) including a predetermined range around the welded portion 201. In this embodiment, welded portion 201 is a so-called weld bead that is formed along a direction along a weld line that is preset by a welding program or the like. The weld line is a virtual line that corresponds to the center line of the weld bead along the longitudinal direction.

[0023] Shape data relating to a plurality of regions 202 including welded portions 201 having similar shapes is stored in the same folder in the first storage unit 11. The first storage unit 11 also stores the shape data after segmentation is performed by the first segmentation performing unit 12.

[0024] In this specification, "segmentation" refers to the process of defining the outline of the welded portion 201 from the shape data of the region 202 and the process of adding annotations to the shape data in which the welded portion 201 is defined. "Annotation" refers to the process of adding information on the presence or absence of a shape defect 210 (see Figures 4B and 5A to 5C) to the shape data. Furthermore, if a shape defect 210 is present, the annotation refers to the process of labeling the shape defect 210 with the type of shape defect, identifying the number, size, and position of the shape defect 210, and adding these identification results to the shape data. Furthermore, annotation data refers to information on the shape defect 210 identified by the annotation. In other words, the annotation data refers to the presence or absence information of the shape defect 210 in the shape data and the identification results of the type, number, size, and position of the shape defect 210. The criteria for determining whether or not the shape data contains the shape defect portion 210 are set in advance based on the required specifications for welding the workpiece 200. In many cases, in order to improve the accuracy of the determination of the weight file, which will be described later, the worker performing the visual inspection visually checks the shape data displayed on the first display device 19 and re-determines whether or not the annotation data output in the weight file is correct.

[0025] That is, the first storage unit 11 stores a plurality of pieces of shape data relating to a plurality of regions 202 including welds 201 having similar shapes, each of which has undergone segmentation. These pieces of shape data are used to generate weight files and reinforce learning.

[0026] The first storage unit 11 may also store other data, such as multiple data obtained by data extension of shape data after segmentation. These data are also used for generating weight files and for strengthening learning, which will be described later.

[0027] The first storage unit 11 also stores a plurality of weight files and various files to be added to the weight files. A weight file is a combination of a plurality of weighted classifiers, and is a known object detection algorithm. For example, it is expressed as a convolutional neural network (CNN), a YOLO (You Only Look Once), or a Faster R-CNN (Regions with Convolutional Neural Networks).

[0028] Specifically, the weight file is a file that describes a group of numerical values ​​required to determine the presence or absence and type of shape defect in the region 202. When shape data is input, the weight file outputs information on the presence or absence of shape defect parts 210 in the shape data, and the results of identifying the type, number, size, and position of the shape defect parts 210.

[0029] In this embodiment, weight files are prepared corresponding to the number n (n is an integer equal to or greater than 2) of types of shape defects included in the shape data of the region 202. That is, as shown in FIG. 2A , n weight files are prepared and stored in the first storage unit 11. For example, the first weight file outputs information on the presence or absence of a first type of shape defect portion 210 among the n types of shape defects, as well as identification results of the type, number, size, and location of the first type of shape defect portion 210. A set of n weight files may be referred to as a judgment model. However, the file structure of the judgment model is not particularly limited thereto. For example, as shown in FIG. 2B , the judgment model may be a set to which a common output file that outputs a final judgment result based on the judgment results of each weight file is further added. When shape data is input, the common output file outputs a judgment result of whether the shape in the region 202 is good or bad based on the judgment results and identification results of each weight file. Note that even in the example shown in FIG. 2B , the judgment process for each weight file is performed independently.

[0030] The first storage unit 11 is configured with semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), SSD (Solid State Drive), etc. The first storage unit 11 may be configured with HDD (Hard Disk Drive), etc. The first storage unit 11 may also be configured in an external server.

[0031] The first segmentation executing unit 12 performs the above-described segmentation on the shape data received by the first data receiving unit 16. In this case, the first segmentation executing unit 12 defines the outline of the welded portion 201 in the shape data and performs processing to identify the welded portion 201 in the shape data, such as processing to color-code the welded portion 201 from other portions or processing to add the outline of the welded portion 201 to the shape data. The first segmentation executing unit 12 also adds annotation data input from the first input device 18 to the shape data. When multiple types of shape defects exist in the shape data, the first segmentation executing unit 12 generates shape data in which segmentation has been performed for each type of shape defect. In other words, the first segmentation executing unit 12 performs processing to define the welded portion 201 in the shape data, and further adds annotations for each type of shape defect, and outputs the results as separate data.

[0032] The first segmentation executing unit 12 may perform a process of defining the welded portion 201 for one piece of shape data and may also simultaneously add annotations for multiple types of shape defect portions 210. In this case, the first segmentation executing unit 12 processes the shape data after the segmentation to generate new shape data for each type of shape defect. This new shape data defines the welded portion 201 and is added with only annotation data for one type of shape defect.

[0033] In addition, when shape data for the same region 202 in which the demarcation process for the welded portion 201 has been performed and shape data to which annotation data related to one type of shape defect has been added are stored separately, the first segmentation executing unit 12 may combine both to generate one piece of shape data. In this specification, shape data in which the demarcation process for the welded portion 201 has been performed and to which annotation data related to only one type of shape defect has been added is referred to as "sub-shape data." The sub-shape data is also stored in the first storage unit 11.

[0034] When multiple pieces of sub-shape data containing the same type of shape defect are input, the weight file generator 13 performs learning using the data as learning data and generates the weight file described above. Note that "learning" in this specification refers to machine learning or deep learning. In this case, the weight file generator 13 generates n weight files shown in FIG. 2A for each type of shape defect. The weight file generator 13 may also generate a judgment model shown in FIG. 2B.

[0035] The weight file generator 13 generates a weight file based on a data file stored in advance in the first storage unit 11. This data file describes the type of shape defect, a judgment threshold, and thresholds related to the number and size of the shape defect portions 210. It also describes a display format for displaying the judgment results based on the weight file on the first display device 19. The judgment threshold is a lower limit of the probability that a portion identified as a shape defect portion 210 by the weight file is a specific type of shape defect. For example, the weight file calculates the probability that the shape defect portion 210 is spatter, and if the probability is equal to or greater than the judgment threshold (e.g., 70%), the type of the shape defect portion 210 is identified as spatter.

[0036] The weight file or judgment model generated by the weight file generator 13 is set in the first judgment unit 14. When the shape data or sub-shape data stored in the first storage unit 11 is input to the first judgment unit 14, it is processed using the weight file or judgment model, and the type of shape defect included in the shape data or sub-shape data, and the number, size, and position of the shape defect portions 210 are output. For example, the output result is displayed on the first display device 19. In this case, the shape defect portions 210 in the shape data or sub-shape data are surrounded by a polygon, circle, or ellipse and are displayed in different colors according to the type of shape defect. However, the present invention is not limited to this, and the type of shape defect, and the number, size, and position of the shape defect portions 210 may be displayed in other display formats.

[0037] The first data control unit 15 controls input and output of various data input to and output from the determination model generating device 10. The first data control unit 15 also controls transmission and reception of various data within the determination model generating device 10.

[0038] The functions of the first segmentation execution unit 12, the weight file generation unit 13, and the first determination unit 14 are realized by executing programs implemented on one or more GPUs. Note that, when realizing the first segmentation execution unit 12, the weight file generation unit 13, and the first determination unit 14, some of the programs described above may be implemented on one or more CPUs. Furthermore, in many cases, the program for realizing the function of the first segmentation execution unit 12 is set separately from the program for realizing the function of the weight file generation unit 13. However, the former program and the latter program may be integrated into one program. Furthermore, the program for realizing the function of the first segmentation execution unit 12 is set separately from the program for realizing the function of the first determination unit 14. However, the former program and the latter program may be integrated into one program.

[0039] The GPU or CPU for realizing the first segmentation execution unit 12 may be provided separately from the GPU or CPU for realizing the weight file generation unit 13. Alternatively, the GPU or CPU for realizing the first segmentation execution unit 12 may be common to the GPU or CPU for realizing the weight file generation unit 13. Alternatively, the GPU or CPU for realizing the first segmentation execution unit 12 and the weight file generation unit 13 may be provided separately from the GPU or CPU for realizing the first determination unit 14. Alternatively, the GPU or CPU for realizing the first segmentation execution unit 12 and the weight file generation unit 13 may be common to the GPU or CPU for realizing the first determination unit 14.

[0040] The first data control unit 15 is configured with one or more CPUs. In many cases, the CPU configuring the first data control unit 15 is provided separately from the CPUs for implementing the first segmentation execution unit 12 and the weight file generation unit 13.

[0041] The first data receiving unit 16 is configured with one or more input ports, and receives shape data transmitted from the appearance inspection device 20. The first data receiving unit 16 also receives data input from the first input device 18. Note that the first data receiving unit 16 may receive data transmitted from a device or server other than the appearance inspection device 20 or the first input device 18.

[0042] The first data transmission unit 17 is configured with one or more output ports, and transmits the judgment result regarding the shape acceptability of the shape data or sub-shape data judged by the weight file or judgment model to the first display device 19. The first data transmission unit 17 also transmits the weight file or judgment model generated by the weight file generation unit 13 to the appearance inspection device 20. The first data transmission unit 17 may transmit data to a device or server other than the appearance inspection device 20 and the first display device 19.

[0043] The first input device 18 is configured, for example, with input devices such as a keyboard and a mouse as shown in Fig. 1. However, the first input device 18 is not particularly limited to this, and may be, for example, an input device with a display function such as a touch panel or a smartphone. In this case, the first input device 18 and the first display device 19 are the same device. The first input device 18 may also include a touch pen.

[0044] A worker performing visual inspection or welding, or both, operates the first input device 18 while viewing the shape data displayed on the first display device 19, and adds annotations to the shape data regarding the shape defect location 210. For example, in the shape data, the worker encloses an area determined to be the shape defect location 210 with a polygon, circle, or ellipse, and inputs the type of shape defect for the enclosed area.

[0045] The operator may also use the first input device 18 to directly write the values ​​used in the weight file into the weight file or into the data file used to generate the weight file. For example, the operator may modify the threshold value for determining shape defects.

[0046] The first display device 19 is configured with a display device such as a liquid crystal display or an organic EL display. The first display device 19 displays the determination result of the first determination unit 14. The display format is saved in an output file, and the content displayed on the first display device 19 can be changed by operating the first input device 18.

[0047] The determination model generating device 10 may also have a first display image processing unit (not shown). The first display device 19 displays the shape data or sub-shape data of the region 202 after segmentation. In this case, the first display image processing unit rotates and displays the shape data or sub-shape data on the first display device 19 in response to an operation from the first input device 18. Alternatively, the first display image processing unit may enlarge or reduce the shape data or sub-shape data in a predetermined direction. In addition, in response to an operation from the first input device 18, the first display image processing unit may color-code the shape defect portion 210 from other portions and display it on the first display device 19. In this case, a comment, for example, the type of shape defect, may be displayed in the color-coded portion.

[0048] The first storage unit 11 and the first determination unit 14 may be provided outside the determination model generating device 10. It is only necessary that the shape data, sub-shape data, weight files, data files constituting the weight files, and common output files can be transmitted and received between the determination model generating device 10 and the first storage unit 11 and the first determination unit 14 via the first data receiving unit 16 and the first data transmitting unit 17.

[0049] [1-2: Configuration of the Visual Inspection Device] The visual inspection device 20 has a shape measurement unit 21, a data pre-processing unit 22, a second segmentation execution unit 23, a second determination unit 24 (hereinafter sometimes referred to as the determination unit 24), a second data control unit 25, a second data receiving unit 26, and a second data transmitting unit 27. The visual inspection device 20 also has a second input device 28 and a second display device 29.

[0050] The appearance inspection device 20 excluding the shape measurement unit 21 is configured with one or more computers, similar to the determination model generation device 10. The computer may have a memory such as an HDD.

[0051] The visual inspection device 20, excluding the shape measurement unit 21, the second input device 28, and the second display device 29, is configured with the above-mentioned multiple processors. However, this is not particularly limited to this, and for example, the device may have a dedicated IC with a noise removal function.

[0052] The shape measurement unit 21 is a three-dimensional shape measurement sensor including a laser light source (not shown) configured to be able to scan the surface of the workpiece 200 and a camera (not shown) that captures the reflection trajectory (sometimes referred to as a shape line) of the laser light projected onto the surface of the workpiece 200. The shape measurement unit 21 is attached to the welding torch 31. The shape measurement unit 21 scans an area 202 including a welding point 201 of the workpiece 200 with a laser beam and captures an image of the laser beam reflected by the area 202 with the camera, thereby measuring the three-dimensional shape of the area 202. The camera has a CCD or CMOS image sensor as an imaging element. However, the configuration of the shape measurement unit 21 is not limited to the above and other configurations may be employed. For example, an optical interferometer may be used instead of the camera.

[0053] The data pre-processing unit 22 has a function of removing noise from the shape data acquired by the shape measurement unit 21. Because the reflectance of the light emitted from the shape measurement unit 21 varies depending on the material of the workpiece 200, if the reflectance is too high, halation or the like may occur, resulting in noise and affecting the shape data. For this reason, the data pre-processing unit 22 is configured to perform noise filtering processing on software. Note that noise can also be removed in a similar manner by providing an optical filter (not shown) in the shape measurement unit 21 itself. High-quality shape data can be obtained by using both the optical filter and software filtering processing.

[0054] The noise removal function of the data preprocessing unit 22 is realized by, for example, a GPU, but is not limited to this and may be realized by, for example, a dedicated IC having a noise removal function.

[0055] The data pre-processing unit 22 also corrects the inclination and distortion of the base portion of the welding point 201 relative to a predetermined reference plane, for example, the installation surface of the workpiece 200, by statistically processing the shape data of the region 202.

[0056] The second segmentation execution unit 23 defines the contour of the welding point 201 for the shape data acquired by the shape measurement unit 21 and subjected to processing such as noise removal by the data pre-processing unit 22. The second segmentation execution unit 23 may perform edge enhancement correction to emphasize the contour of the welding point 201 in order to emphasize the shape and position of the welding point 201.

[0057] The second determination unit 24 determines whether the shape of the region 202 including the welded portion 201 is good or not, based on the shape data of the region 202 in which the outline of the welded portion 201 is defined by the second segmentation execution unit 23 and the weight file or the determination model generated by the determination model generation device 10. In other words, the second determination unit 24 determines whether the shape data of the region 202 acquired by the shape measurement unit 21 satisfies a predetermined determination criterion.

[0058] As described above, a weight file is provided for each type of shape defect. Therefore, basically, each of the multiple weight files generated by the judgment model generating device 10 is set in the second judgment unit 24. However, as will be described later, in this embodiment, the presence or absence of a shape defect is judged sequentially for each type of shape defect. Therefore, when judging the presence or absence of one type of shape defect, the weight file corresponding to that type of shape defect is set in the second judgment unit 24.

[0059] In this state, when the shape data of the region 202 processed by the second segmentation executing unit 23 is input to the second judging unit 24, a judgment result as to whether or not the shape data of the region 202 satisfies a predetermined judgment criterion for one type of shape defect, as well as details of the judgment result, are output. Note that whether or not to output the details of the judgment result, such as the number, size, and position of the shape defect points 210 in the region 202, is set in advance in the weight file. Note that the common output file shown in FIG. 2B may have a function for outputting details of the judgment results for all types of shape defects. In this case, the function for outputting details of the judgment result may not be incorporated into each weight file.

[0060] 1, the weight file or the judgment model is set in the second judgment unit 24 from the first storage unit 11 of the judgment model generating device 10 via the second data control unit 25. However, this is not particularly limited to this, and for example, a second storage unit (not shown) may be provided in the appearance inspection device 20, multiple types of weight files or judgment models may be stored in the second storage unit, and the weight files or judgment models may be set in the second judgment unit 24 when the appearance inspection is performed.

[0061] The functions of the second segmentation executing unit 23 and the second determination unit 24 are realized by executing programs implemented on one or more GPUs. Note that, when realizing the second segmentation executing unit 23 and the second determination unit 24, part of the programs described above may be implemented on one or more CPUs. Furthermore, in many cases, the program for realizing the function of the second segmentation executing unit 23 is set separately from the program for realizing the function of the second determination unit 24. However, the former program and the latter program may be shared.

[0062] The GPU or CPU for realizing the second segmentation executing unit 23 may be provided separately from the GPU or CPU for realizing the second determination unit 24. Furthermore, the GPU or CPU for realizing the second segmentation executing unit 23 may be common to the GPU or CPU for realizing the second determination unit 24. In other words, the second segmentation executing unit 23 may be a functional block common to the second determination unit 24 and executed by common hardware.

[0063] The second data control unit 25 controls input and output of various data input to and output from the appearance inspection device 20. The second data control unit 25 also controls transmission and reception of various data within the appearance inspection device 20.

[0064] The second data control unit 25 is composed of one or more CPUs. In many cases, the CPU constituting the second data control unit 25 is provided separately from the CPUs for realizing the second segmentation execution unit 23 and the second determination unit 24.

[0065] The second data receiving unit 26 is configured with one or more input ports and receives a weight file, a judgment model, or annotation data transmitted from the judgment model generating device 10. The second data receiving unit 26 may also receive a data file for determining shape defects. The second data receiving unit 26 also receives data input from the second input device 28. The second data receiving unit 26 may also receive data transmitted from a device or server other than the judgment model generating device 10 or the second input device 28.

[0066] The second data transmission unit 27 is composed of one or more output ports, and transmits the judgment result regarding the shape quality of the shape data judged by the weight file or judgment model set in the second judgment unit 24 to the second display device 29. Note that the second data transmission unit 27 may transmit data to a device or server other than the judgment model generating device 10 and the second input device 28.

[0067] The hardware configuration of the second input device 28 is the same as that of the first input device 18. For example, the worker operates the second input device 28 to change part of the weight file, for example, a data file that describes the judgment criteria, etc. For example, the worker may modify the judgment threshold value for shape defects.

[0068] Furthermore, when correcting the weight file, the user operates the second input device 28 while looking at the shape data in which the judgment defect occurred, and adds an annotation relating to the shape defect portion 210 to the shape data.

[0069] The hardware configuration of the second display device 29 is the same as that of the first display device 19. The second display device 29 displays the judgment result regarding the shape acceptability of the shape data judged by the weight file or judgment model set in the second judgment unit 24. The display format is saved in the weight file or common output file, and the content displayed on the second display device 29 can be changed by operating the second input device 28.

[0070] The appearance inspection device 20 may also have a second display image processing unit (not shown). In this case, for example, the second display device 29 may display shape data of the region 202 after segmentation has been performed. In response to an operation from the second input device 28, the second display image processing unit rotates and displays the shape data on the second display device 29. Alternatively, the second display image processing unit may enlarge or reduce the shape data in a predetermined direction and display it. In response to an operation from the second input device 28, the second display image processing unit may color-code the shape defect portion 210 from other portions and display it on the second display device 29. In this case, a comment, for example, the type of shape defect, may be displayed in the color-coded portion.

[0071] The data preprocessing unit 22 and the second determination unit 24 may be provided outside the appearance inspection device 20. It is only necessary that the shape data, sub-shape data, weight files, and determination models can be transmitted and received between the appearance inspection device 20 and the data preprocessing unit 22 and the second determination unit 24 via the second data receiving unit 26 and the second data transmitting unit 27.

[0072] [1-3: Configuration of Welding Apparatus] Welding apparatus 30 has welding torch 31, a wire feeder (not shown), welding power source 32, output control unit 33, robot arm 34, and robot control unit 35.

[0073] When power is supplied from the welding power source 32 to the welding wire 36 held by the welding torch 31, an arc is generated between the tip of the welding wire 36 and the workpiece 200, and the workpiece 200 is heated to perform arc welding. Note that the welding device 30 has other components and equipment such as piping and gas cylinders for supplying shielding gas to the welding torch 31, but for the sake of convenience, these are not shown or described.

[0074] Output control unit 33 is connected to welding power source 32 and a wire feeder (not shown) and controls the welding output of welding torch 31 in accordance with predetermined welding conditions, in other words, the power supplied to welding wire 36 and the power supply time. Output control unit 33 also controls the feed speed and feed amount of welding wire 36 fed from the wire feeder (not shown) to welding torch 31. Note that the welding conditions may be input directly to output control unit 33 via an input unit (not shown), or may be selected from a welding program separately read from a recording medium or the like.

[0075] Robot arm 34 is a known articulated robot that holds welding torch 31 at its tip and is connected to robot control unit 35. Robot control unit 35 controls the operation of robot arm 34 so that the tip of welding torch 31, in other words, the tip of welding wire 36 held by welding torch 31, moves to a desired position while tracing a predetermined welding trajectory.

[0076] [2: Findings that led to the present disclosure] Fig. 3 is a schematic plan view of shape data of a region including a welded portion. Fig. 4A is a schematic plan view of the shape data shown in Fig. 3 after the contours of the welded portion have been defined. Fig. 4B is a schematic plan view of the shape data shown in Fig. 4A after annotations have been added. Fig. 5A is an enlarged view of a portion in Fig. 4B where spatters exist. Fig. 5B is an enlarged view of a portion in Fig. 4B where holes exist. Fig. 5C is an enlarged view of a portion in Fig. 4B where multiple types of shape defects exist.

[0077] As described above, the shape data is acquired by the shape measurement unit 21 and processed by the data preprocessing unit 22, such as noise removal, to obtain shape data of the region 202, as shown in FIG.

[0078] As described above, segmentation is performed on the shape data shown in Fig. 3 . For example, while viewing the image of the region 202 displayed on the second display device 29, the worker operates the second input device 28 to define the outline of the welded portion 201 and add that information to the image as shown in Fig. 4A . Furthermore, the worker operates the second input device 28 to surround the locations of shape defects, and as shown in Fig. 4B , each surrounded region is color-coded and the type of shape defect is identified. In other words, annotation data, which is information identifying the position, number, size, and type of the shape defect 210, is added to the image of the region 202.

[0079] In the example shown in Fig. 4B, there are multiple geometrically defective portions 210 in the region 202, and there are also multiple types of geometrical defects. At one end of the region 202 shown in Fig. 4B, there are multiple spatters 211, as shown in Fig. 5A. The spatters 211 are metal particles that fly off the workpiece 200 or the welding wire 36 during arc welding and adhere to the surface of the workpiece 200.

[0080] 4B, a hole (also called a perforated portion) 212 is formed to the right of the portion where multiple spatters 211 are present (see FIG. 5B). The hole 212 is a hole formed by penetrating the workpiece 200 during arc welding. Furthermore, at the other end of the region 202 shown in FIG. 4B, multiple types of shape defects 210 are present, as shown in FIG. 5C. Specifically, in addition to the spatters 211 described above, a pit 213 and an undercut 214 are present. The pit 213 is a pore that opens on the surface of the workpiece 200 during arc welding. The undercut 214 is a groove formed by digging into the workpiece 200 along the end of the welded portion 201 during arc welding.

[0081] When visual inspection of region 202 is performed by visual inspection device 20, the results of defining the contour of welded portion 201 and specifying its position and size in region 202 (hereinafter referred to as contour data) shown in Fig. 4A and the annotation data shown in Fig. 4B are added to the shape data shown in Fig. 3. Furthermore, the presence or absence of a shape defect in region 202, as well as the type and position of the shape defect, are identified using a weight file or a judgment model, and finally, the quality of the shape is judged.

[0082] Furthermore, the shape data to which various data after the appearance inspection has been added is transmitted to the determination model generating device 10 as learning data for generating a weight file and for reinforcing learning, and is stored in the first storage unit 11. The first storage unit 11 stores a plurality of learning data and corresponding weight files for each material and shape of the workpiece 200 and for each welding method.

[0083] However, the inventors of the present application have found that when learning data is accumulated sequentially and the weight file is trained and strengthened using the accumulated multiple learning data, the detection rate of shape defects in visual inspection does not improve, or the degree of improvement is small.

[0084] As a result of further investigation, it was estimated that if multiple types of shape defects are annotated simultaneously to one training data, a weight file is generated based on this training data, and training is strengthened, the recognition rate of shape defects may actually decrease for one or multiple types of shape defects.

[0085] Therefore, the inventors of the present application have proposed preparing a weight file for each type of shape defect and determining the presence or absence of a shape defect in the region 202 using multiple weight files, and have found that this actually improves the detection rate of shape defects in visual inspection. This will be further explained below.

[0086] [3: Procedure for Generating Weight File] Fig. 6 is a flowchart showing the procedure for generating a weight file according to embodiment 1. Fig. 7 is a schematic plan view of shape data when contour data of a welding portion is added.

[0087] To generate a weight file, it is first necessary to obtain multiple pieces of training data. Therefore, workpieces 200 on which welds 201 are formed are prepared. At least two types of workpieces 200 are prepared, each having the same shape and material and with welds 201 of the same shape formed in the same position. One type is a workpiece 200 that does not include a shape defect 210 and will be referred to as a non-defective product in the following description. The other type is a workpiece 200 that includes a shape defect 210 and will be referred to as a defective product in the following description. Whether or not the aforementioned region 202 in the workpiece 200 includes a shape defect 210 is determined by an operator in accordance with predetermined criteria, as described above. It is also preferable to prepare at least multiple defective products.

[0088] Next, shape data is acquired for each of the non-defective and defective products (step S1). This shape data is acquired by the visual inspection device 20. However, the shape data may be acquired separately using another device.

[0089] Segmentation is performed on the non-defective product data, which is shape data of the non-defective product, to obtain contour data of the welding point 201 (step S2). As the contour data, only the position of the contour of the welding point 201 in the shape data may be obtained as point cloud data, or, for example, point cloud data of the contour and the area surrounded by the contour may be obtained as shown in Fig. 7.

[0090] Next, in the defective product data, which is the shape data of the defective product, an annotation is first added to only one type of shape defect, and annotation data regarding that type of shape defect is obtained (step S3).

[0091] Steps S2 and S3 are executed by the first segmentation execution unit 12 of the decision model generating device 10. In addition, when executing steps S2 and S3, the operator uses the first input device 18 and the first display device 19 as necessary.

[0092] The contour data acquired in step S2 and the annotation data acquired in step S3 are combined to generate sub-shape data (step S4). A plurality of sub-shape data are generated based on the annotation data acquired from a plurality of defective products. The plurality of sub-shape data are used to generate a weight file and reinforce learning. The process of step S4 is executed by the first segmentation execution unit 12.

[0093] A weight file corresponding to the type of shape defect in the sub-shape data generated in step S4 is prepared (step S5). In step S5, a weight file generated in advance by another device and stored in the first storage unit 11 may be read and executed, or a weight file may be generated by the weight file generator 13 using some of the multiple sub-shape data.

[0094] Next, the weight file prepared in step S5 is set in the weight file generator 13, and when shape data is input, the appearance evaluation result of the region 202 is output (step S6). The input shape data indicates that the welded portion 201 is formed at the same position as the sub-shape data generated in step S4. This shape data is read from the first storage unit 11, acquired by the appearance inspection device 20, or input to the judgment model generating device 10 from an external device via the first data receiver 16.

[0095] The shape evaluation result is displayed on the first display device 19. Specifically, as described above, the shape defect portion 210 of the designated type in the shape data is displayed on the first display device 19 surrounded by a polygon, a circle, or an ellipse.

[0096] While viewing the image of the shape data displayed on the first display device 19, the worker checks whether the actual shape defect portion 210 has been correctly determined by the weight file (step S7). Specifically, the worker checks whether a shape defect of the specified type exists in the enclosed area in the shape data. The worker also checks whether the size of the enclosed area is too large or too small compared to the size of the actual shape defect portion 210. The worker also checks whether a shape defect of a different type from the specified type has been recognized as a shape defect of the specified type.

[0097] If the confirmation result in step S7 is positive, that is, if the type, number, size, and position of the shape defect are correctly determined or specified by the weight file, the weight file is confirmed (step S8).

[0098] On the other hand, if the confirmation result of step S7 is negative, that is, if at least one of the type, number, size, and position of the shape defect determined or identified by the weight file is incorrect, re-learning is performed on the weight file (step S9). The number of re-learning attempts and the amount of learning data used in step S9 may be changed depending on the content of the misjudgment of the weight file. For example, if the weight file misjudged the type of shape defect, re-learning is further strengthened. Furthermore, the operator re-annotates the shape data confirmed in step S7 to generate new sub-shape data. This sub-shape data is used for re-learning in step S9. After executing step S9, the process returns to step S6, where shape evaluation of the shape data is performed. In this case, new shape data is input into the weight file. Furthermore, the series of processes of steps S6, S7, and S9 are repeatedly executed until the confirmation result of step S7 becomes positive and the weight file is finalized.

[0099] Once the weight file corresponding to one type of shape defect has been determined, the process returns to step S3, and annotations for the next type of shape defect are added to the shape data of the defective product. Thereafter, the series of processes from steps S4 to S8 or S4 to S9 described above are executed sequentially. The series of processes from steps S3 to S8 or S3 to S9 are executed until the corresponding weight files for all types of shape defects that have been confirmed in advance, that is, for all n types of shape defects, are determined.

[0100] When the weight files corresponding to the n types of shape defects have been determined, the process is completed.

[0101] Although not shown, after the n types of weight files are determined, a common output file may be associated with each of the n types of weight files to generate the determination model shown in FIG. 2B.

[0102] 4: Procedure for determining whether the shape of a region including a welded portion is acceptable Fig. 8 is a flowchart showing the procedure for determining whether the shape of a region including a welded portion is acceptable Fig. 9 is an example of an output image of the determination result of the shape of the region.

[0103] First, the welded workpiece 200 is set in the visual inspection device 20 (step S11), and the area 202 including the welded portion 201 is measured by the shape measurement unit 21 to obtain shape data for the area 202 (step S12).

[0104] 6 , a first weight file of the n weight files generated in the procedure shown in FIG. 6 is set in the second determination unit 24, and the shape data acquired in step S12 is input to the first weight file (step S13). The first weight file set in the second determination unit 24 determines whether the shape data of the region 202 includes a first type of shape defect (step S14). In step S14, the probability that the shape defect detected in the first weight file is the first type, for example, spatter 211, is calculated, and if the probability is equal to or greater than a determination threshold value (for example, 70%), the detected shape defect is determined to be spatter 211.

[0105] The second judging section 24 also identifies the number, size, and position of the first type of shape defect portions 210 in the region 202 (step S15).

[0106] When step S15 is completed, the process returns to step S13, where a second type of weight file out of the n types of weight files is set in the second determination unit 24, and the shape data acquired in step S12 is input into the second type of weight file. The series of processes from steps S13 to S15 is repeatedly executed until shape data is input into all of the n types of weight files, the presence or absence of corresponding types of shape defects is determined, and the number, size, and position of shape defect locations 210 are identified.

[0107] The final shape pass / fail judgment result for the region 202 is compiled in a predetermined format and output from the second judgment unit 24, and then transmitted to the second display device 29 or an external device via the second data transmission unit 27 (step S16). The output result transmitted from the second judgment unit 24 is displayed on the second display device 29, for example, as shown in FIG. 9. However, the format of the output result shown in FIG. 9 is merely an example and is not particularly limited to this format. For example, the shape pass / fail judgment result and the type and number of shape defects may be simply output and displayed. Furthermore, the scale of the shape data shown in FIG. 9 is merely an example. It may be changed as appropriate depending on the size of the welded portion 201 and the minimum detectable size of the shape measurement unit 21. For example, as shown in FIG. 9, the scale may be 100 μm or 0.1 μm. Furthermore, the common output file shown in FIG. 2B may be used to obtain the output result shown in FIG. 9.

[0108] In addition, if the shape quality judgment results of all areas 202 contained in one workpiece 200 are good in light of predetermined judgment criteria, the workpiece 200 is judged to be a good product and is sent to the subsequent processing process or shipped as a good product.

[0109] On the other hand, if the shape quality judgment result is determined to be defective for one or more regions 202 included in one workpiece 200, several measures can be taken. For example, after performing a visual inspection of all regions 202 included in the workpiece 200, the inspection results are saved and the workpiece 200 is discarded as a defective product. Also, when a defect is found in the visual inspection of the region 202, the workpiece 200 may be discarded as a defective product.

[0110] Also, for example, after performing a visual inspection of all the regions 202 included in the workpiece 200, the inspection results may be saved and the workpiece 200 may proceed to a repair process. In the repair process, rewelding is performed on the welded points 201 included in the regions 202 determined to be defective.

[0111] [5: Weight File Update Method] FIG. 10 is a flowchart showing a weight file update method.

[0112] As mentioned above, if the same weight file is used to continuously judge the shape of the area 202 including the welded area 201, the reliability of the judgment result (hereinafter referred to as judgment reliability) will decrease, and in extreme cases, it may become impossible to judge a shape defect that was previously possible to judge.

[0113] There are various possible causes for this, but the following is the most likely one. That is, as the weight file learning progresses each time a visual inspection is performed, the weighting values ​​within the weight file, which were once optimized, change. If the balance of the values ​​set within the weight file is significantly disrupted at this time, it is presumed that this will cause a decrease in the reliability of shape determination.

[0114] Therefore, in this embodiment, the weight file is updated according to the procedure shown in Fig. 10. The weight file generation method in this specification also includes as part thereof the update method shown in Fig. 10. This will be further explained below.

[0115] First, the shape of the region 202 is evaluated using each of the n weight files (step S21). Typically, step S21 is performed during a visual inspection of the region 202 including the welded portion 201. That is, step S21 is the same as repeating the processes of steps S13 to S15 shown in FIG. 8 n times. However, this is not limited to this, and may be, for example, the processes of steps S341 to S36n in FIG. 15 shown later. Furthermore, the process of step S21 may be performed separately from the visual inspection of the region 202.

[0116] Next, the result of the shape evaluation performed in step S21 is judged (step S22). In step S22, it is not judged whether or not there is a shape defect, but rather whether or not the shape evaluation itself has been performed correctly. For example, it is judged whether or not there is a so-called undetected state, in which a shape defect that should be detected has not been detected, or whether or not there is a so-called overdetected state, in which a shape defect is detected in a place where there is no shape defect portion 210. The judgment of whether or not the shape evaluation itself has been performed correctly is usually made by an operator.

[0117] If the determination result in step S22 is affirmative, that is, if the shape evaluation itself has been performed correctly, the process ends.

[0118] On the other hand, if the determination result in step S22 is negative, that is, if the shape evaluation itself has not been performed correctly, the processes from step S23 onward are executed.

[0119] In step S23, the weight file whose determination reliability is below a predetermined value and the type of shape abnormality corresponding to this weight file are identified. Furthermore, sub-shape data for which the determination reliability is below the predetermined value are extracted. In this embodiment, the number of shape defects identified in step S23 is defined as m (m≦n).

[0120] Next, the annotation data of the sub-shape data is corrected for one type of shape defect among the m types of shape defects, in this case the jth shape defect (j is an integer, j = 1, ..., m) (step S24). The process of step S24 is performed on the sub-shape data to which annotation data has been added for the jth shape defect among the shape data evaluated in step S22. Specifically, if there is an undetected shape defect as described above for the jth shape defect, an annotation is added to this shape defect location 210, and the annotation is deleted for the location determined to be the shape defect location 210 in an overdetected state. The process of step S24 is usually performed by an operator.

[0121] The sub-shape data whose annotation data has been corrected in step S24 is added to the learning data, and additional learning is performed on the weight file related to the j-th shape defect (step S25). That is, multiple sub-learning data related to the j-th type of shape defect, including the sub-shape data corrected in step S24, are input to the j-th weight file, and additional learning is performed on the j-th weight file. After step S25 is performed, the shape of the region 202 is evaluated using the j-th weight file (step S26).

[0122] Next, the result of the shape evaluation performed in step S26 is judged (step S27). In step S27, as in step S22, it is judged whether the shape evaluation itself has been performed correctly. As in step S22, the judgment of whether the shape evaluation itself is correct or not is also made by the operator.

[0123] If the determination result in step S27 is positive, that is, if the shape evaluation itself has been performed correctly, the j-th weight file is determined (step S28).

[0124] On the other hand, if the judgment result of step S27 is negative, that is, if the shape evaluation itself has not been performed correctly, the process returns to step S24. The series of processes from steps S24 to S27 is repeated until the judgment result of step S27 becomes positive, in other words, until the judgment reliability of the j-th shape defect using the j-th weight file becomes equal to or greater than a predetermined value. Here, the predetermined value of the judgment reliability can be determined arbitrarily by the operator. In other words, the predetermined value of the judgment reliability does not need to be constant.

[0125] Subsequently, a series of processes from steps S24 to S27 are executed until the weight files corresponding to all types of shape defects identified in step S23, i.e., j types of shape defects, are determined. When the weight files corresponding to j types of shape defects are determined, the process ends.

[0126] [6: Effects, etc.] As described above, the weight file generation method according to this embodiment includes at least the following steps 1 to 5. When shape data, which is three-dimensional shape data of a predetermined area 202 including a welding point 201, is input, the weight file outputs at least the presence or absence of a specific type of shape defect in the shape data and the type of shape defect.

[0127] In the first step (step S1 in FIG. 6 ), good product data and defective product data are acquired based on predetermined criteria. The good product data is shape data relating to good products that do not contain any shape defects in the region 202. The defective product data is shape data relating to defective products that contain n types of shape defects in the region 202 (n is an integer of 2 or greater, and is the number of types of shape defects contained in the region 202).

[0128] In the second step (step S2 in FIG. 6), contour data of the welded portion 201 in the non-defective product data is obtained.

[0129] In the third step (step S3 in FIG. 6), the number, size, position and type of shape defect at the shape defect parts 210 are identified, and the identification results for each type of shape defect are obtained as annotation data.

[0130] In the fourth step (step S4 in FIG. 6), the contour data and annotation data are combined to generate sub-shape data for each type of shape defect.

[0131] In the fifth step (steps S5 to S9 in FIG. 6), n types of weight files are determined for each type of sub-shape data.

[0132] The weight file of one type is generated and determined to determine the presence or absence of the shape defect of one type in the region 202 and to identify the number, size, and location of the shape defect portions 210 of one type in the region 202 .

[0133] In addition, the weight file generation method of this embodiment includes a sixth step (steps S21 to S28 in Figure 10) of additionally learning one type of weight file when the reliability of determining one type of shape defect using one type of weight file falls below a predetermined value.

[0134] According to this embodiment, it is possible to improve the reliability of determining shape defects using a weight file compared to conventional methods, which will be further explained below with reference to the drawings.

[0135] Fig. 11 is a plan view schematic diagram of shape data when annotations related to multiple types of shape defects are simultaneously added. Fig. 12A is a plan view schematic diagram of shape data when annotations related only to sputtering are added. Fig. 12B is a plan view schematic diagram of shape data when annotations related only to holes are added. Fig. 12C is a plan view schematic diagram of shape data when annotations related only to pits are added.

[0136] 11 and 12A to 12C, sputters 211 are illustrated as circles or ellipses, holes 212 are illustrated as triangles, and pits 213 are illustrated as rectangles. Also, the dashed lines surrounding each of the shape-defective portions 210 indicate that the shape-defective portions 210 have been annotated.

[0137] According to the conventional method, as shown in FIG. 11, annotations are added to one piece of shape data for all types of shape defects contained therein, and the shape data after annotations are added is used as learning data for a weight file.

[0138] However, with conventional methods, as the number of training data increases and weight file training progresses, the number of cases where a certain type of shape defect cannot be detected tends to increase, i.e., the proportion of undetected shape defects of a certain type tends to increase. In other words, even if the training of the weight file is strengthened, the judgment reliability may plateau or, in extreme cases, decrease. Furthermore, if a shape defect that actually exists is judged to be undetected, the proportion of erroneous shape defect reduction measures taken or shape defect repairs made using an incorrect method increases, making it impossible to improve the welding yield of the workpiece 200.

[0139] 12A to 12C, in this embodiment, annotations are added to one type of shape defect contained in one piece of shape data to generate sub-shape data, and the sub-shape data is used as learning data for a weight file for evaluating the one type of shape defect described above.

[0140] By doing so, it is possible to reduce the rate of erroneous determination of the presence or absence of shape defects when determining using the weight file, thereby improving the reliability of determination compared to conventional methods.

[0141] 15 is a diagram showing the reliability of shape defect determination when using weight files generated by the conventional method and the method shown in embodiment 1. The vertical axis of the graph represents the confidence value.

[0142] A weight file whose learning has been reinforced using learning data generated by a conventional method is referred to as a conventional weight file, and a weight file whose learning has been reinforced using sub-shape data generated by the method shown in this embodiment as learning data is referred to as a weight file of this embodiment.

[0143] 15, the confidence value, which is the degree of reliability of determining shape defects, was improved when the weight file of this embodiment was used compared to when the conventional weight file was used. Furthermore, regardless of the type of shape defect, the confidence value was improved when the weight file of this embodiment was used compared to when the conventional weight file was used.

[0144] 15, when the type of shape defect is a hole, the confidence value when the conventional weight file is used is 81.0%, while the confidence value when the weight file of this embodiment is used is 88.3%. Also, when the type of shape defect is a sputter, the confidence value when the conventional weight file is used is 84.8%, while the confidence value when the weight file of this embodiment is used is 87.7%.

[0145] Furthermore, according to this embodiment, by providing the sixth step described above, the judgment reliability is improved only for weight files whose judgment reliability has fallen below a predetermined value through additional learning, so that the judgment reliability of other weight files that do not undergo additional learning can be maintained at a high value without fluctuating.

[0146] Fig. 13 is a plan view schematic diagram of shape data when annotations related to multiple types of shape defects are simultaneously added and corrected. Fig. 14A is a plan view schematic diagram of shape data when annotations related only to undetected spatter are added. Fig. 14B is a plan view schematic diagram of shape data when annotations related only to overdetected spatter are corrected. Note that Figs. 14A and 14B explain the example of only spatter as a shape defect, but annotation data is also corrected in the same way for holes 212, pits 213, undercuts 214, and other types of shape defects if there are undetected or overdetected spatters.

[0147] 13, if there is an undetected or overdetected state of a shape defect, the annotation data is usually corrected regardless of the type of shape defect. The corrected shape data is used as learning data for the weight file.

[0148] However, when shape data is corrected using this method and added as learning data, as mentioned above, there may be an increase in cases where one type of shape defect is mistakenly determined to be a different type of shape defect.

[0149] Therefore, in this embodiment, a sixth step is provided after the fifth step, in which one type of weight file is additionally learned only when the determination reliability of one type of shape defect using that type of weight file falls below a predetermined value. In this way, the determination reliability can be improved by additional learning only for weight files whose determination reliability falls below the predetermined value. Furthermore, the determination reliability of other weight files that are not subjected to additional learning can be maintained at a high value without fluctuating.

[0150] The sixth step preferably includes a seventh step (step S23 in FIG. 10), an eighth step (step S24 in FIG. 10), and a ninth step (step S25 in FIG. 10).

[0151] In the seventh step, sub-shape data is extracted when the determination reliability of one type of shape defect falls below a predetermined value.

[0152] In the eighth step, the annotation data in the sub-shape data extracted in the seventh step is corrected. Specifically, as shown in Fig. 14A, if a spatter 211, which is an undetected shape defect, is confirmed, the area where the spatter 211 exists is defined and an annotation is added. Also, as shown in Fig. 14B, if a spatter 211 is confirmed in a portion where a shape defect portion 210 does not actually exist, the annotation data for this portion is deleted because it is an overdetection state.

[0153] In the ninth step, a plurality of sub-learning data relating to one type of shape defect, including the sub-shape data corrected in the eighth step, is input into one type of weight file, and the one type of weight file is additionally learned.

[0154] The series of processes from the seventh step to the ninth step are repeatedly executed until the reliability of the determination of one type of shape defect reaches a predetermined value or more.

[0155] By doing so, it is possible to reliably improve the determination reliability by additional learning only for weight files whose determination reliability falls below a predetermined value.

[0156] In this embodiment, a weight file is determined sequentially for each type of shape defect. That is, in the third step, the identification result for a first type of shape defect among n types of shape defects is obtained as first annotation data. In the fourth step, the outline data and the first annotation data are combined to generate first sub-shape data. In the fifth step, a first weight file corresponding to the first type of shape defect is determined.

[0157] The series of processes from the third step to the fifth step are repeated until all of the n types of weight files are determined.

[0158] In this way, for example, the determination model generating device 10 can be realized by a single computer, which means that the cost of building the determination model generating device 10 can be kept low.

[0159] The fifth step includes at least a tenth step (step S5 in FIG. 6), an eleventh step (step S6 in FIG. 6), and a twelfth step (step S7 in FIG. 6).

[0160] In step 10, a first weight file corresponding to the first type of shape defect is prepared. In step 11, shape data is input into the first weight file, and an appearance evaluation is performed on the region 202. In step 12, it is confirmed whether the evaluation result in step 11 is correct.

[0161] If the result of the check in step 12 is affirmative, the first weight file is confirmed. If the result of the check in step 12 is negative, step 5 further executes step 13 (step S9 in FIG. 6 ) of inputting the first sub-shape data into the first weight file and re-learning the first weight file.

[0162] A series of processes including steps 5, 10, 11, 12 and 13 are repeatedly executed until the first weight file is determined.

[0163] By doing so, learning is strengthened for each of the n types of weight files, and the reliability of the determination can be further improved.

[0164] The weight file is generated by causing one or more processors to execute a dedicated program that describes the procedure shown in FIG.

[0165] That is, the weight file generating program according to this embodiment causes one or more processors to execute the first to thirteenth steps described above.

[0166] The method for visually inspecting the welded portion 201 of the workpiece 200 according to this embodiment includes at least the following sixteenth to twentieth steps.

[0167] In a sixteenth step (step S12 in FIG. 8), three-dimensional shape data of a predetermined region 202 including the welding point 201 is acquired as shape data.

[0168] In the seventeenth step (step S13 in FIG. 8), shape data is input into the weight file generated in the procedure shown in FIG.

[0169] In the eighteenth step (step S14 in FIG. 8), the presence or absence of a shape defect in the region 202 is determined based on the weight file.

[0170] In a nineteenth step (step S15 in FIG. 8), the number, size, and location of the shape defect portions 210 in the region 202 are identified.

[0171] In a twentieth step (step S16 in FIG. 8), the acceptability of the shape of the region 202 is determined based on the determination result and the identification result using the weight file.

[0172] According to the visual inspection method of this embodiment, the weight file generated by the above-mentioned generation method is used, so that the shape of the area 202 including the welded portion 201 can be determined with high reliability.

[0173] In this embodiment, in step 17, shape data is input into a first weight file corresponding to a first type of shape defect among the n types of shape defects. In step 18, the presence or absence of the first type of shape defect is determined. In step 19, the number, size, and position of the first type of shape defect portions 210 in the region 202 are identified.

[0174] Furthermore, the series of processes from step 17 to step 19 are repeatedly executed for all of the n types of shape defects. In step 20, the shape of the region 202 is judged to be good or bad based on the judgment results and identification results using the n types of weight files.

[0175] The shape of the area 202 including the welded portion 201 is determined by executing a dedicated program that describes the procedure shown in FIG. 8 on one or more processors.

[0176] That is, the program for determining whether the shape of the region 202 is good or bad according to this embodiment causes one or more processors to execute the above-mentioned sixteenth to twentieth steps.

[0177] In this way, for example, it is possible to reduce the number of computers included in the visual inspection device 20 to one. In other words, it is possible to keep the cost of constructing the visual inspection device 20 low.

[0178] The visual inspection apparatus 20 according to this embodiment includes at least a shape measurement unit 21 and a second determination unit 24 (determination unit 24). As described above, the visual inspection apparatus 20 may also include other functional blocks and devices.

[0179] The shape measurement unit 21 measures the three-dimensional shape of an area 202 including the welding location.

[0180] The weight files generated by the above-described generation method are set in the second determination unit 24. The shape data acquired by the shape measurement unit 21 is input to each weight file, and the second determination unit 24 (determination unit 24) determines whether the shape of the region 202 is good or bad.

[0181] According to the appearance inspection device 20 of this embodiment, the weight file generated by the above-mentioned generation method is used, so that the shape of the area 202 including the welded portion 201 can be determined with high reliability.

[0182] In addition, by inputting shape data into one type of weight file, the second judgment unit 24 judges whether or not one type of shape defect exists in the region 202, and identifies the number, size, and position of one type of shape defect location 210 in the region 202.

[0183] Furthermore, the second determination unit 24 determines the presence or absence of shape defects across all of the n types of shape defects, and specifies the number, size, and position of the shape defect parts 210. In other words, the second determination unit 24 determines whether the shape of the region 202 is good or bad based on the determination result of the presence or absence of the n types of shape defects in the region 202 and the specification result of the number, size, and position of the n types of shape defect parts 210 in the region 202.

[0184] In this way, it is possible to determine in detail whether the shape of the area 202 including the welded portion 201 is good or bad with high reliability.

[0185] (Embodiment 2) Fig. 16 is a flowchart showing the procedure for generating a weight file according to embodiment 2. Fig. 17 is a flowchart showing the i-th subroutine. Note that steps S31 and S32 in Fig. 16 are similar to steps S1 and S2 in Fig. 6, respectively, and therefore will not be described.

[0186] Furthermore, when the reliability of determining a type of shape defect using a type of weight file falls below a predetermined value, the weight file is updated in the procedure shown in Fig. 10 , as in embodiment 1. That is, as in embodiment 1, the method of generating a weight file includes both the procedure shown in Fig. 16 and the procedure shown in Fig. 10 .

[0187] The method for generating a weight file according to this embodiment shown in Fig. 16 differs from the method for generating a weight file according to the first embodiment shown in Fig. 6 in the following respects: First, in step S33 of Fig. 16, annotations for each of n types of shape defects are added in parallel to the shape data of the region 202, and annotation data is acquired.

[0188] In step S34, annotation data for each of the n types of shape defects is combined with the contour data to generate n types of sub-shape data in parallel. As in the first embodiment, these sub-shape data become learning data for the weight file.

[0189] After preparing n types of weight files in step S35, n subroutines are executed in parallel, and the n types of weight files are determined in parallel. In this embodiment, the determination model generation device 10 is configured with n computers, and one subroutine is executed and processed by one computer.

[0190] As shown in Fig. 17, the processing of steps S36i to S39i in the i-th subroutine is the same as the processing of steps S6 to S9 shown in Fig. 6. In other words, step S35 and the first to n-th subroutines shown in Fig. 16 correspond to the fifth step described above, and n types of weight files are prepared in parallel for each type of sub-shape data. Furthermore, for each type of shape defect, sub-shape data is input into a weight file, and learning is reinforced in parallel for the n types of weight files.

[0191] According to this embodiment, n types of weight files are determined in parallel, so the time required to generate the weight files can be significantly reduced compared to the case where n types of weight files are determined sequentially as shown in embodiment 1. Depending on the time required for reinforced learning in each weight file, the time required to generate all the weight files can be reduced to approximately 1 / n at most.

[0192] Furthermore, according to this embodiment, it is possible to achieve the same effects as those achieved by the configuration shown in embodiment 1. That is, it is possible to improve the reliability of shape defect determination using weight files compared to conventional methods. Furthermore, by strengthening learning for each of the n types of weight files, it is possible to further improve the reliability of determination.

[0193] The weight file generation program according to this embodiment causes a processor to execute the first to thirteenth steps described above, except that the fifth step is one of the first to n-th subroutines described above, and in this embodiment, at least the fifth step is executed using multiple processors.

[0194] Next, the appearance inspection method of this embodiment will be described. Fig. 18 is a flowchart showing the procedure for determining whether the shape of a region including a welded portion is acceptable or not according to embodiment 2. Steps S41 and S42 in Fig. 18 are the same as steps S11 and S12 in Fig. 8, respectively, and therefore will not be described again.

[0195] The appearance inspection method of this embodiment shown in Fig. 18 differs from the appearance inspection method of embodiment 1 shown in Fig. 8 in the following respects: First, in step S43 of Fig. 18, the shape data acquired in step S42 is copied to generate n pieces of shape data.

[0196] Next, the processes of steps S13 to S15 shown in FIG. 8 are executed in parallel for each type of shape defect.

[0197] In this embodiment, the appearance inspection device 20 excluding the shape measurement unit 21 is composed of n computers, and the processing of steps S13 to S15 for one type of shape defect is executed by one computer.

[0198] That is, in the visual inspection method of this embodiment, the processes of the above-mentioned 17th to 20th steps are executed as follows.

[0199] In the 17th step (steps S441 to S44n in FIG. 18), shape data is input into each of the n types of weight files. In the 18th step (steps S451 to S45n in FIG. 18), the presence or absence of each of the n types of shape defects is determined. In the 19th step (steps S461 to S46n in FIG. 18), the number, size, and position of each of the n types of shape defect locations 210 in the region 202 are identified. In the 20th step (step S47 in FIG. 18), the acceptability of the shape of the region 202 is determined based on the determination results and identification results using the n types of weight files.

[0200] According to this embodiment, the appearance evaluation and pass / fail judgment are performed in parallel for each of the n types of shape defects in the region 202. Therefore, the time required for the appearance inspection can be significantly reduced, or even reduced to about 1 / n, compared to the case where the appearance evaluation and pass / fail judgment are performed sequentially for each of the n types of shape defects in the region 202 as shown in the first embodiment.

[0201] Depending on the value of n, some types of shape defects may be inspected in the manner shown in FIG. 8, and the remaining types of shape defects may be inspected in the manner shown in FIG.

[0202] Furthermore, according to this embodiment, it is possible to achieve the same effects as those achieved by the configuration shown in embodiment 1. In other words, because n types of weight files with improved judgment reliability are used, it is possible to judge with high reliability whether the shape of the region 202 including the welded portion 201 is good or bad. Furthermore, it is possible to improve the judgment reliability by additional learning only for weight files whose judgment reliability is below a predetermined value. Furthermore, it is possible to maintain a high value without changing the judgment reliability of other weight files for which additional learning is not performed.

[0203] Furthermore, the program for determining whether the shape of the region 202 is good or bad according to this embodiment causes a processor to execute the above-mentioned steps 16 to 20. In this embodiment, at least steps 17 to 20 are executed using multiple processors.

[0204] (Embodiment 3) Fig. 19 is a flowchart showing a procedure for generating a weight file according to embodiment 3. Fig. 20 is a schematic plan view showing an example of data extension processing. Steps S51 to S53 and steps S56 to S60 in Fig. 19 are similar to steps S1 to S3 and steps S6 to S8 in Fig. 6, respectively, and therefore will not be described.

[0205] Furthermore, when the reliability of determining a type of shape defect using a type of weight file falls below a predetermined value, the weight file is updated in the procedure shown in Fig. 10 , as in embodiment 1. That is, as in embodiment 1, the method of generating a weight file includes both the procedure shown in Fig. 19 and the procedure shown in Fig. 10 .

[0206] The method for generating a weight file according to this embodiment shown in Fig. 19 differs from the method for generating a weight file according to the first embodiment shown in Fig. 6 in the following respects: First, the flowchart shown in Fig. 19 includes step S54 (fourteenth step) between step S53 (third step) and step S55 (fourth step).

[0207] In step S54, annotation data related to one type of shape defect is subjected to data augmentation. In step S55, the contour data and the annotation data augmented in step S54 are combined to generate sub-shape data related to one type of shape defect. The sub-shape data generated in step S55 is used to generate a weight file related to one type of shape defect and for strengthening learning. In step S55, multiple sub-shape data are generated, and the number of sub-shape data generated is appropriately set depending on the number of sub-shape data required to generate the corresponding weight file and for strengthening learning. For example, the number of data-augmented annotation data and the number of sub-shape data generated in step S55 may be changed for each type of shape defect.

[0208] Here, the processing of step S54 will be further explained. In Fig. 20, data extension is performed by changing the positions and number of sputters 211 in the region 202, and three different types of annotation data are generated by performing data extension on the original annotation data.

[0209] Generally, data expansion is performed by changing one or more features in the shape data of region 202, or by changing the position, number, or size of each shape defect point 210 for multiple types of shape defects, or by doing both.

[0210] Here, the feature amount refers to a specific parameter extracted from the shape data, and typical examples include the length, width, and height from a reference plane of the welded portion 201, as well as differences in length, width, and height between multiple points within the welded portion 201. Therefore, when changing the size of the shape defect portion 210, the size can be changed not only in a direction parallel to the surface of the workpiece 200 on which the welded portion 201 is formed, but also in a direction perpendicular to the surface of the workpiece 200, for example, in the height direction of the weld bead. Furthermore, the size ratio can also be changed in each of the directions parallel and perpendicular to the surface of the workpiece 200. However, the feature amount is not particularly limited to these, and is set appropriately depending on the content determined for each inspection item.

[0211] Note that data expansion is not particularly limited to that shown in Fig. 20. For example, a larger amount of annotation data may be generated by data expansion. Similarly, data expansion is performed to generate a larger amount of annotation data for holes 212, pits 213, undercuts 214, and other types of shape defects.

[0212] As described above, in step S54, the number of annotation data related to one type of shape defect increases significantly due to data expansion. Accordingly, the number of sub-shape data generated in step S55 also increases significantly compared to the case shown in embodiment 1. When the annotation data generated by data expansion is combined with the contour data in step S55, the shape defect locations 210 are automatically partitioned by color or the like. In other words, for the specified type of shape defect, the sub-shape data is generated with the number, size, and position of the shape defect locations 210 in the region 202 specified.

[0213] Thus, according to this embodiment, by performing data expansion processing to increase the number of annotation data related to one type of shape defect, it is possible to significantly increase the number of learning data used to generate one type of weight file and to strengthen learning.

[0214] Conventionally, to increase the number of learning data, it was necessary to prepare multiple samples in which welding was performed at the same location on the same type of workpiece 200. Furthermore, it was necessary to acquire shape data for each sample, perform segmentation to define the outline of the welded location 201, and add annotations for each type of shape defect.

[0215] However, generating a weight file for one type of shape defect and strengthening the learning generally requires hundreds to tens of thousands of pieces of learning data, and the conventional methods mentioned above require a huge amount of work and cost to acquire the learning data.

[0216] On the other hand, according to this embodiment, by data-extending annotation data and further combining it with the data-extended annotation data of the contour data, a large amount of sub-shape data can be generated in a short time. In other words, the amount of learning data used for generating weight files and strengthening learning can be easily increased, and the judgment reliability of weight files can be improved in a short time. Furthermore, the judgment reliability of only weight files whose judgment reliability falls below a predetermined value can be improved by additional learning. Furthermore, the judgment reliability of other weight files that do not undergo additional learning can be maintained at a high value without fluctuating.

[0217] Furthermore, the weight file generation program according to this embodiment causes one or more processors to execute the first to fourteenth steps described above, except that the fifth step is one of the first to n-th subroutines described above, and in this embodiment, at least the fifth step is executed using multiple processors.

[0218] (Fourth embodiment) Fig. 21 is a flowchart showing a procedure for generating a weight file according to the fourth embodiment. Fig. 22 is a flowchart showing the i-th subroutine.

[0219] Steps S61 to S63 and step S66 in Fig. 21 are similar to steps S31 to S33 and step S35 in Fig. 16, respectively, and therefore will not be described further. Also, the flowchart in Fig. 22 showing the specific processing of each of the first to nth subroutines in Fig. 21 is similar to the flowchart shown in Fig. 17, and therefore will not be described further.

[0220] Furthermore, when the reliability of determining a type of shape defect using a type of weight file falls below a predetermined value, the weight file is updated in the procedure shown in Fig. 10 , as in embodiment 1. That is, as in embodiment 1, the method of generating a weight file includes both the procedure shown in Fig. 21 and the procedure shown in Fig. 10 .

[0221] The method for generating a weight file according to this embodiment shown in Fig. 21 differs from the method for generating a weight file according to the second embodiment shown in Fig. 16 in the following respects: First, the flowchart shown in Fig. 21 includes step S64 (15th step) between step S63 (third step) and step S65 (fourth step).

[0222] In step S64, annotation data related to one type of shape defect is expanded, as in the third embodiment. In step S65, the contour data and the annotation data expanded in step S54 are combined to generate sub-shape data related to one type of shape defect. The sub-shape data generated in step S65 is used to generate a weight file related to one type of shape defect and to reinforce learning.

[0223] This embodiment can achieve the same effects as the configuration shown in embodiment 2. That is, because n types of weight files are determined in parallel, the time required to generate the weight files can be significantly reduced, and more specifically, to about 1 / n of the time required in embodiment 1. In particular, when strengthening the learning of each weight file, the effect of reducing the generation time is remarkable.

[0224] Furthermore, according to this embodiment, it is possible to achieve the same effects as those achieved by the configuration shown in embodiment 3. That is, by data-extending annotation data and further combining contour data with the data-extended annotation data, it is possible to generate a large amount of sub-shape data in a short time. That is, it is possible to easily increase the amount of learning data used to generate weight files and reinforce learning, and it is possible to improve the judgment reliability of weight files in a short time. Furthermore, it is possible to improve the judgment reliability by additional learning only for weight files whose judgment reliability falls below a predetermined value. Furthermore, it is possible to maintain a high value without changing the judgment reliability of other weight files that do not undergo additional learning.

[0225] In this embodiment, the judgment model generating device 10 is configured with n computers, and one subroutine is executed by one computer, as in the second embodiment. The processes of steps S53 to S55 may also be executed by one computer for one type of shape defect.

[0226] In other words, the weight file generation program according to this embodiment executes the first to ninth and fifteenth steps described above using one or more processors. However, the fifth step is one of the first to nth subroutines described above, and in this embodiment, at least the fifth step is executed using multiple processors. It is also preferable that the processes of steps S63 to S65 are executed using multiple processors.

[0227] (Other Embodiments) The components shown in embodiments 1 to 4 can be combined as appropriate to create new embodiments. For example, when a weight file generated by the method shown in embodiments 1 and 3 is set in second determination unit 24 of visual inspection device 20, visual inspection of area 202 including welded portion 201 may be performed according to the procedure shown in Fig. 18. In this way, the time required to generate the weight file can be significantly reduced compared to the case shown in embodiment 1.

[0228] Furthermore, when a weight file generated by the method shown in embodiments 2 and 4 is set in the second judgment unit 24 of the visual inspection device 20, a visual inspection of the area 202 including the welding point 201 may be performed using the procedure shown in Figure 8.

[0229] Furthermore, in the present specification, the welding device 30 is shown as an example of a device that performs arc welding, but is not particularly limited to this, and may be a welding device 30 that performs other types of welding, such as laser welding or ultrasonic welding.

[0230] Furthermore, although the present specification has described the visual inspection of the region 202 including the welded portion 201, the present specification is not particularly limited to this. For example, the weight file generated by the generation method disclosed in the present specification can be similarly applied to the visual inspection of the region 202 including the processed portion after laser processing such as laser cutting or machining has been performed. The visual inspection method disclosed in the present specification can also be applied. In this case, too, the reliability of determining shape defects using the weight file can be improved. Furthermore, the shape of the region 202 including the processed portion can be determined with high reliability. In other words, "welding" in the present specification can be read as "processing."

[0231] The weight file generation method disclosed herein obtains a weight file that can improve the reliability of determining whether the shape of a specified area including a weld is acceptable, and is useful for applying to the visual inspection of the area.

[0232] REFERENCE SIGNS LIST 10 Determination model generating device 11 First storage unit 12 First segmentation executing unit 13 Weight file generating unit 14 First determination unit 15 First data control unit 16 First data receiving unit 17 First data transmitting unit 18 First input device 19 First display device 20 Visual inspection device 21 Shape measuring unit 22 Data pre-processing unit 23 Second segmentation executing unit 24 Second determination unit (determination unit) 25 Second data control unit 26 Second data receiving unit 27 Second data transmitting unit 28 Second input device 29 Second display device 30 Welding device 31 Welding torch 32 Welding power source 33 Output control unit 34 Robot arm 35 Robot control unit 36 ​​Welding wire 100 Welding system 200 Work 201 Welding point 202 Area 210 Shape defect area 211 Spatter 212 Hole 213 Pit 214 Undercut

Claims

1. A weight file generation method for receiving shape data representing three-dimensional shape data of a predetermined region including a welded portion and outputting at least the presence or absence of a specific type of shape defect in the shape data, the weight file generation method comprising at least the following steps: a first step of acquiring good product data representing the shape data of good products that do not include the shape defect in the region, and defective product data representing the shape data of defective products that include n types of shape defects in the region (n is an integer of 2 or greater and is the number of types of shape defects included in the region); a second step of acquiring contour data of the welded portion in the good product data; a third step of identifying the type of shape defect in the shape defect portion and acquiring the identification results for each type of shape defect as annotation data; a fourth step of combining the contour data and the annotation data to generate sub-shape data for each type of shape defect; a fifth step of determining the n types of weight files for each type of sub-shape data; and a sixth step of additionally learning the one type of weight file.

2. A weight file generation method as defined in claim 1, wherein the sixth step includes a seventh step, an eighth step, and a ninth step, wherein the seventh step extracts the sub-shape data when the reliability of the judgment result of the one type of shape defect falls below a predetermined value, the eighth step corrects the annotation data in the sub-shape data extracted in the seventh step, and the ninth step inputs a plurality of sub-learning data related to the one type of shape defect, including the sub-shape data corrected in the eighth step, into the one type of weight file to additionally learn the one type of weight file, and the series of processes from the seventh step to the ninth step are repeatedly executed until the reliability of the judgment result of the one type of shape defect becomes equal to or greater than the predetermined value.

3. A weight file generating method as claimed in claim 1, wherein in the third step, the identification result for a first type of shape defect among the n types of shape defects is used as first annotation data; in the fourth step, the contour data and the first annotation data are combined to generate first sub-shape data; and in the fifth step, a first weight file corresponding to the first type of shape defect is determined; and a series of processes from the third step to the fifth step are repeatedly executed until all of the n types of weight files have been determined.

4. A weight file generating method as defined in claim 3, wherein the fifth step includes at least a tenth step, an eleventh step, and a twelfth step, wherein the tenth step prepares a first weight file corresponding to the first type of shape defect, the eleventh step inputs the shape data into the first weight file and performs an appearance evaluation of the area, and the twelfth step checks whether the evaluation result of the tenth step is correct, and if the check result of the twelfth step is positive, the first weight file is finalized, and if the check result of the twelfth step is negative, the fifth step further includes a thirteenth step of inputting the first sub-shape data into the first weight file and re-learning the first weight file, and wherein the series of processes from the tenth step to the thirteenth step are repeatedly executed until the first weight file is finalized.

5. A weight file generation method as claimed in claim 1, characterized in that in the third step, the annotation data relating to the n types of shape defects are acquired in parallel, respectively; in the fourth step, the n types of sub-shape data are generated in parallel, respectively; and in the fifth step, the n types of weight files are determined in parallel, respectively.

6. A weight file generation method as claimed in claim 5, characterized in that in the fifth step, the n types of weight files are prepared in parallel for each type of sub-shape data, and further, the sub-shape data is input into the weight file for each type of shape defect, and learning of the n types of weight files is reinforced in parallel.

7. A weight file generating method as described in claim 3, further comprising a fourteenth step between the third step and the fourth step, wherein the first annotation data is data-extended in the fourteenth step, and the first sub-shape data is generated in the fourth step by combining the contour data with the first annotation data data-extended in the fourteenth step.

8. A weight file generating method according to claim 5, further comprising a fifteenth step between the third step and the fourth step, wherein the fifteenth step extends the annotation data, and the fourth step combines the contour data and the annotation data extended in the fifteenth step for each type of shape defect, thereby generating the n types of sub-shape data in parallel.

9. A weight file generation method according to claim 1, characterized in that the weight file of said one type is generated to determine the presence or absence of said shape defect of said one type in said area, and to identify the number, size and position of said shape defect locations of said one type in said area.

10. A method for visual inspection of a workpiece using a weight file generated by the method for generating a weight file according to any one of claims 1 to 9, comprising at least the following steps: a 16th step of acquiring the shape data; a 17th step of inputting the shape data into the weight file; an 18th step of determining whether or not the shape defect exists in the area using the weight file; a 19th step of identifying the number, size and position of the shape defect points in the area; and a 20th step of determining whether or not the shape of the area is good based on the results of the determination and identification using the weight file.

11. A visual inspection method as defined in claim 10, wherein in the 17th step, the shape data is input into a first weight file corresponding to a first type of shape defect among the n types of shape defects; in the 18th step, the presence or absence of the first type of shape defect is determined; in the 19th step, the number, size and position of the first type of shape defect locations in the area are identified; a series of processes from the 17th step to the 19th step are repeatedly performed for all of the n types of shape defects; and in the 20th step, the quality of the shape of the area is determined based on the determination results and the identification results using the n types of weight files.

12. A visual inspection method as claimed in claim 10, characterized in that in the 17th step, the shape data is input into each of the n types of weight files; in the 18th step, the presence or absence of each of the n types of shape defects is determined; in the 19th step, the number, size and position of each of the n types of shape defect locations in the area are identified; and in the 20th step, the quality of the shape of the area is determined based on the determination results and the identification results using the n types of weight files.

13. An appearance inspection device comprising at least a shape measurement unit and a judgment unit, wherein the shape measurement unit measures the three-dimensional shape of the area including the welded area, and the judgment unit is set with n types (n is an integer of 2 or more) of weight files for judging shape defects of the three-dimensional shape included in a predetermined area including the welded area, and the shape data is input into each of the n types of weight files, so that the judgment unit judges whether the shape of the area is good or bad, wherein the n types of weight files are provided for each type of shape defect, and the n types of weight files are set in a state where one type of weight file can be additionally learned.

14. An appearance inspection device as described in claim 13, wherein, when the shape data is input into the weight file of the one type, the judgment unit judges whether or not the one type of shape defect exists in the area, and identifies the number, size, and position of the one type of shape defect location in the area; and further, the judgment unit judges whether or not the shape of the area is good based on the judgment result of the presence or absence of the n types of shape defects in the area and the identification result of the number, size, and position of the n types of shape defect location in the area.

Images