Learning data acquisition device, and learning data acquisition method

Abstract

To provide a learning data acquisition device and a learning data acquisition method capable of suppressing a deterioration in the prediction accuracy of a model and reducing time and effort required to acquire learning data.SOLUTION: In a welding system 1, an AI model generation device 2 being a learning data acquisition device includes an uncorrected image data storage part 211 for storing image data of a 3D image acquired by photographing a welding place by a 3D sensor 36, and a processor 22 for specifying a point at which a value of a Z coordinate satisfies a prescribed condition in the image data stored in the uncorrected image data storage part 211 as a data loss point, correcting the Z coordinate of the data loss point in the image data stored in the uncorrected image data storage part 211, and acquiring the corrected image data as image data to be used for machine learning.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a learning data acquisition device and a learning data acquisition method for acquiring image data used in machine learning. [Background technology]

[0002] Patent Document 1 discloses that when creating a model by machine learning using training data, training data determined to be inappropriate is excluded from the training data stored in a training data storage means. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 187594 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, it is possible to use a model created by machine learning to make various predictions about 3D images obtained by photographing with a 3D sensor. In order to create such a model, learning data consisting of a plurality of data sets of image data of a 3D image photographed by a 3D sensor and annotation data of a prediction result about the 3D image shown by the image data is required. However, when acquiring image data by a 3D sensor, depending on the shape, state, angle to the 3D sensor, etc. of the surface irradiated with the laser from the 3D sensor, the laser may not return to the 3D sensor, resulting in data missing points in the acquired image data. If image data including such data missing points is used as learning data, there is a risk that the prediction accuracy of the created model will deteriorate.

[0005] However, if image data including missing data points is excluded from the learning data used for machine learning, as in Patent Document 1, it becomes necessary to repeatedly capture images using a 3D sensor until image data without missing data points is obtained, which makes it time-consuming to obtain the learning data.

[0006] The present invention has been made in consideration of the above-mentioned points, and an object of the present invention is to suppress deterioration in the prediction accuracy of the model and to reduce the effort required to obtain training data. [Means for solving the problem]

[0007] In order to achieve the above object, the present disclosure provides a learning data acquisition process for acquiring image data to be used in machine learning to create a model, the process comprising storing image data of a 3D image acquired by photographing a welding point with a 3D sensor in an image data storage unit, identifying a point in the image data stored in the image data storage unit whose Z coordinate value satisfies a predetermined condition as a data missing point, correcting the Z coordinate of the data missing point in the image data stored in the image data storage unit, and acquiring the corrected image data as image data to be used in the machine learning.

[0008] This allows the image data to be corrected when it contains missing data points, and by using the corrected image data for machine learning, it is possible to suppress deterioration in the predictive accuracy of the model caused by the image data used for machine learning containing missing data points.

[0009] In addition, since image data containing missing data points can be corrected and the corrected image data can be used as training data, there is no need to repeatedly take images using the 3D sensor until training data without missing data points is obtained, thereby reducing the effort required to obtain training data. Effect of the Invention

[0010] According to the present disclosure, it is possible to suppress deterioration in the predictive accuracy of a model and reduce the effort required to obtain training data. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a welding system including an AI model generation device as a learning data acquisition device according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a flowchart illustrating the operation of the AI ​​model generation device for acquiring learning data. [Diagram 3] FIG. 3 is a diagram illustrating a 3D image represented by image data in which a defect has occurred. [Figure 4] FIG. 4 is an enlarged view of the area around the defect in FIG. [Diagram 5] FIG. 5 is a plan view showing the vicinity of a defective region in a 3D image represented by pre-corrected image data on an XY coordinate plane. [Figure 6] FIG. 6 is a diagram corresponding to FIG. 5 in a first example in which the point at the end of the defective region in the Y-axis direction is located at the edge of the XY coordinate plane represented by the image data. [Figure 7] FIG. 7 is a diagram corresponding to FIG. 5 in a first example in which the point at the end of the defective region in the Y-axis direction is located at the edge of the XY coordinate plane represented by the image data. [Figure 8] FIG. 8 is a side view illustrating data missing points on the YZ coordinate plane after linear interpolation. [Figure 9] FIG. 9 is a diagram equivalent to FIG. 8, in which there are two pairs of two points that sandwich the defective region in the Y-axis direction and have a common X-coordinate. [Figure 10] FIG. 10 is a diagram illustrating an example of an image output by the first output device in a state in which the type and location of a welding defect are designated by a user's input operation. [Figure 11] FIG. 11 is a view corresponding to FIG. 1 of the second embodiment. [Figure 12] FIG. 12 is a perspective view illustrating an example of a shape of a welding defect represented by 3D data stored in a defect shape storage unit. [Figure 13] FIG. 13 is a plan view of the sputter shown in FIG. [Figure 14] FIG. 14 is a cross-sectional view taken along line XIV-XIV in FIG. [Figure 15] FIG. 15 is a cross-sectional view taken along line XV-XV in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its application, or its uses.

[0013] (Embodiment 1) Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.

[0014] 1 shows a welding system 1. This welding system 1 includes an AI model generation device 2 serving as a learning data acquisition device according to a first embodiment of the present disclosure, and a welding device 3 that performs welding.

[0015] The AI ​​model generation device 2 acquires image data that constitutes learning data used in machine learning to create a defect detection model.

[0016] The defect detection model is an object detection model that predicts and outputs welding defect information based on input image data. The defect detection model is represented by a CNN (Convolutional Neural Network). The welding defect information includes category information indicating the type of welding defect in a 3D image represented by the image data, and position information indicating the position of the welding defect. In detail, the category information indicates whether the welding defect is a hole, a pit, a spatter, an undercut, a protrusion, or the like. A pit is an opening on the surface of a weld bead. The position information indicates the position of a bounding box including the welding defect. The bounding box is a rectangular boundary area that surrounds an object such as a welding defect.

[0017] In addition, the AI ​​model generation device 2 acquires annotation data of welding defect information, and generates a defect detection model by machine learning using a plurality of data sets of the acquired image data and the annotation data as learning data.

[0018] Specifically, the AI ​​model generation device 2 includes a storage device 21, a processor 22 as a processing unit, a first output device 23, and a first input device 24.

[0019] The storage device 21 includes a pre-corrected image data storage unit 211 , a post-corrected image data storage unit 212 , an annotation data storage unit 213 , and a parameter storage unit 214 .

[0020] The pre-correction image data storage unit 211 stores, as pre-correction image data, image data of a 3D image acquired by photographing a welding point with a 3D sensor 36 described later.

[0021] The corrected image data storage unit 212 stores the corrected image data obtained by the processor 22 correcting the uncorrected image data.

[0022] The annotation data storage unit 213 stores therein defective welding information specified by a user's input to the first input device 24 in a state in which an image based on the corrected image data is output to the first output device 23.

[0023] The parameter storage unit 214 stores parameters for specifying a fault detection model generated by the processor 22, which will be described later.

[0024] The processor 22 has an image data receiving unit 221, a missing point identifying unit 222, a correction unit 223, an annotation data receiving unit 224, and an AI model generating unit 225.

[0025] The image data receiving unit 221 receives image data of a 3D image obtained by photographing the welding point using the 3D sensor 36 described later from the welding device 3, and stores it in the pre-correction image data storage unit 211 as pre-correction image data.

[0026] The missing point identifying unit 222 identifies, in the pre-corrected image data stored in the pre-corrected image data storage unit 211, a point whose Z coordinate value satisfies a predetermined condition as a data missing point. Here, the predetermined condition is that the value is -999.999. Therefore, the missing point identifying unit 222 identifies, as a data missing point, a point whose Z coordinate value is -999.999. Depending on the specifications of the image data, a point that satisfies a predetermined condition other than the condition that the value is -999.999 may be identified as a data missing point.

[0027] The correction unit 223 corrects the Z coordinate of the data missing point identified by the missing point identification unit 222 in the pre-correction image data stored in the pre-correction image data storage unit 211 to a value in a state where there is no welding defect. Then, the correction unit 223 stores the corrected image data in the post-correction image data storage unit 212. Details of the correction method by the correction unit 223 will be described later.

[0028] The annotation data receiving unit 224 outputs an image based on the corrected image data stored in the corrected image data storage unit 212 to the first output device 23. In this state, an input signal corresponding to a user's input operation is received from the first input device 24. Here, the first input device 24 receives the user's input operation for specifying the type and position of the welding defect in the image. When the user's input operation is completed, the annotation data receiving unit 224 stores the welding defect information identified based on the user's input operation in the annotation data storage unit 213 as annotation data of the welding defect information.

[0029] The AI ​​model generation unit 225 creates a fault detection model by performing machine learning using learning data consisting of multiple sets of data pairs of image data stored in the corrected image data storage unit 212 and annotation data stored in the annotation data storage unit 213. Specifically, the AI ​​model generation unit 225 specifies the weight and bias of each node constituting the CNN expressing the fault detection model as parameters specifying the fault detection model. Then, the AI ​​model generation unit 225 stores the specified parameters in the parameter storage unit 214.

[0030] In this way, the processor 22 acquires the image data corrected by the correction unit 223 as image data to be used for machine learning by the AI ​​model generation unit 225.

[0031] The first output device 23 outputs an image based on the corrected image data stored in the corrected image data storage unit 212. The first output device 23 is configured by, for example, a liquid crystal monitor.

[0032] The first input device 24 receives an input operation by the user specifying the type and position of a welding defect in the image, and outputs an input signal according to the received input operation.

[0033] The welding device 3 has a welding torch 31, a wire feeder (not shown), a welding power source 32, an output control unit 33, a robot arm 34, a robot control unit 35, a 3D sensor 36, a computer 37, a second output device 38, and a second input device 39. When power is supplied from the welding power source 32 to the welding wire WI held by the welding torch 31, an arc is generated between the tip of the welding wire WI and the workpiece W, and the workpiece W is heated to perform arc welding. Note that the welding device 3 has other components and equipment such as piping and gas cylinders for supplying shielding gas to the welding torch 31, but for convenience of explanation, these are not shown and will not be described.

[0034] Output control unit 33 is connected to welding power source 32 and a wire feeder (not shown) and controls the welding output of welding torch 31, in other words, the power supplied to welding wire WI and the power supply time, in accordance with predetermined welding conditions. Output control unit 33 also controls the feed speed and feed amount of welding wire WI fed from a wire feeder (not shown) to welding torch 31. Note that the welding conditions may be input directly to output control unit 33 via an input unit (not shown), or may be selected from a welding program separately read out from a recording medium or the like.

[0035] Robot arm 34 is a known multi-joint robot, holds welding torch 31 at its tip, and is connected to robot control unit 35. Robot control unit 35 controls the operation of robot arm 34 so that the tip of welding torch 31, in other words, the tip of welding wire WI held by welding torch 31, draws a predetermined welding trajectory and moves to a desired position.

[0036] The 3D sensor 36 is attached to the welding torch 31 and measures the shape of the welded portion PW of the workpiece W. The 3D sensor 36 is a three-dimensional shape measurement sensor that is composed of, for example, a laser light source (not shown) configured to be able to scan the surface of the workpiece W, and a camera (not shown) that captures the reflection trajectory of the laser light (hereinafter, sometimes referred to as a shape line) projected on the surface of the workpiece W. The 3D sensor 36 scans the entire welded portion PW of the workpiece W with a laser beam, and captures the laser beam reflected by the welded portion PW with the camera, thereby measuring the shape of the welded portion PW. The 3D sensor 36 is configured to perform shape measurement not only on the welded portion PW but also on a predetermined range around it. This is for evaluating the presence or absence of spatters, etc. The camera has a CCD or CMOS image sensor as an imaging element. The configuration of the 3D sensor 36 is not particularly limited to the above, and other configurations may be adopted. For example, an optical interferometer may be used instead of the camera.

[0037] The computer 37 executes software implemented on a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) to realize the functions of a plurality of functional blocks in the computer 37. The computer 37 has an image processing unit 371, a welding defect information calculation unit 372, and a welding defect display image generation unit 373.

[0038] The image processing unit 371 receives the shape data acquired by the 3D sensor 36 and converts it into image data of a 3D image including the welding point PW. For example, the image processing unit 371 acquires point cloud data of the shape lines captured by the 3D sensor 36. The image processing unit 371 also corrects the inclination and distortion of the base portion of the welding point PW with respect to a predetermined reference plane, for example, the installation plane of the workpiece W, by statistically processing the point cloud data, and acquires image data including the welding point PW.

[0039] The defective welding information calculation unit 372 calculates (predicts) defective welding information based on the image data generated by the image processing unit 371, using the defect detection model generated by the AI ​​model generation device 2. The calculation of the defective welding information may be performed when a predetermined input operation is performed on the second input device 39.

[0040] The poor welding display image generating unit 373 calculates a pixel value of each pixel of the poor welding display image displaying the poor welding information, based on the poor welding information calculated by the poor welding information calculating unit 372. Information identifying the poor welding display image may be compressed by a JPEG format or the like.

[0041] The second output device 38 displays the poor welding display image based on the pixel value of each pixel calculated by the poor welding display image generating unit 373. The second output device 38 is configured by a liquid crystal monitor or the like.

[0042] The second input device 39 accepts a predetermined input operation by a user for causing the computer 37 to start calculating the defective welding information.

[0043] Next, the operation of acquiring learning data by the AI ​​model generation device 2 will be described with reference to the flowchart of FIG.

[0044] First, in (S101), the 3D sensor 36 captures an image of the welding spot PW, and the image processing unit 371 receives the shape data acquired by the 3D sensor 36 and acquires image data of the 3D image.

[0045] Next, in (S102), the image data receiving unit 221 of the AI ​​model generation device 2 receives the image data acquired in (S101) from the welding device 3 and stores it in the pre-correction image data storage unit 211 as pre-correction image data.

[0046] Next, in (S103), the missing point identifying unit 222 identifies, as a data missing point, a point whose Z coordinate value is -999.999 in the pre-corrected image data stored in (S102) in the pre-corrected image data storage unit 211. In other words, the missing point identifying unit 222 identifies a missing area on the XY coordinate plane consisting of the XY coordinates of the data missing points (a distribution area of ​​the data missing points on the XY coordinate plane).

[0047] The image data acquired in (S101) may have defects, for example, as shown by the curve SL in Fig. 3 and Fig. 4. The Z coordinate value of the data defect point is -999.999. Causes of defects include the shape of the laser irradiation surface and the angle of the irradiation surface with respect to the 3D sensor 36, which causes the laser emitted from the 3D sensor 36 and hitting the irradiation surface to not be reflected in the direction of the 3D sensor 36, and the laser emitted from the 3D sensor 36 being absorbed by slag attached to the laser irradiation surface.

[0048] Next, the correction unit 223 corrects the Z coordinate of the data missing point to a value in a state where there is no welding defect by the following processes (S104) to (S106).

[0049] First, in (S104), the correction unit 223 identifies adjacent points adjacent in the Y-axis direction to the missing area identified in (S103). Fig. 5 shows on the XY coordinate plane the area around the missing area in the 3D image shown by the pre-correction image data stored in the pre-correction image data storage unit 211. In Fig. 5, the data missing points are indicated by oblique hatching. In the example of Fig. 5, the correction unit 223 identifies points P11 to P15 and P21 to P25 as the adjacent points.

[0050] As shown in Figs. 6 and 7, when the point at the end of the missing area in the Y-axis direction is located at the end of the XY coordinate plane shown by the image data, the correction unit 223 specifies the point at the end of the missing area in the Y-axis direction as an end point. In the example of Fig. 6, the correction unit 223 specifies points P31 to P34 as adjacent points of the missing area, and specifies points P41 to P44 as end points of the missing area. In the example of Fig. 7, the correction unit 223 specifies points P51 to P56 and P61 to P63 as adjacent points of the missing area, and specifies points P64 to P66 as end points of the missing area. In addition, the correction unit 223 specifies a reference point that is closest to each end point in the X-axis direction and has a common position in the Y-axis direction. When there are two points that are closest to each end point in the X-axis direction and have a common position in the Y-axis direction, the point with the smaller X coordinate value is set as the reference point. In the example of Fig. 6, the point PR is specified as the reference point of the end points P41 to P44. In the example of FIG. 7, point P61 is specified as a reference point for end points P64 and P65, and point P62 is specified as a reference point for end point P66.

[0051] Next, in (S105), the correction unit 223 determines the equation of a linear function for calculating the Z coordinate based on the Y coordinate for each two points that sandwich the missing area from the Y axis direction and have a common X coordinate among the adjacent points identified in (S104). That is, it determines the coefficients a and b of the equation of the linear function expressed as Z = aY + b. The equation of the linear function can be calculated based on the Y coordinate and Z coordinate of the two points that have a common X coordinate among the adjacent points identified in (S104).

[0052] For adjacent points that do not have adjacent points with a common X coordinate, such as points P31 to P34 in Fig. 6 and points P52 to P54 in Fig. 7, the equation of a linear function for calculating the Z coordinate based on the Y coordinate is determined using the Y and Z coordinates of the adjacent points, the Y coordinate of an end point with a common X coordinate with the adjacent points, and the Z coordinate of a reference point of the end point. For example, for adjacent point P31 in Fig. 6, the equation of the linear function is determined based on the Y and Z coordinates of adjacent point P31, the Y coordinate of end point P41, and the Z coordinate of reference point PR. For adjacent point P53 in Fig. 7, the equation of the linear function is determined based on the Y and Z coordinates of adjacent point P53, the Y coordinate of end point P65, and the Z coordinate of reference point P61.

[0053] Next, in (S106), the correction unit 223 calculates the Z coordinate value of each data missing point by providing the Y coordinate value of the data missing point to the equation of the linear function specified in (S105). The calculation of the Z coordinate value of the data missing point is performed using the equation of the linear function determined for the adjacent point whose X coordinate is equal to that of the data missing point. For example, in FIG. 7, the calculation of the Z coordinate value of the data missing point whose X coordinate is equal to that of the adjacent point P53 uses the equation of the linear function determined based on the Y coordinate and Z coordinate of the adjacent point P53, the Y coordinate of the end point P65, and the Z coordinate of the reference point P61. Then, the correction unit 223 corrects the Z coordinate of each data missing point (point whose XY coordinate is located in the missing area) in the pre-correction image data stored in the pre-correction image data storage unit 211 to the calculated Z coordinate value. Then, the correction unit 223 proceeds to the process of (S107).

[0054] By the processes of (S104) to (S106), for example, as shown in FIG. 8, the Z coordinate of the data missing point between points P15 and P25 in FIG. 5 is linearly interpolated so that the data missing point is located on the straight line connecting points P15 and P25 on the YZ coordinate plane.

[0055] 9, there are two pairs of points that sandwich the missing area in the Y-axis direction and have a common X-coordinate. In this example, the Z coordinate of the data missing point between points PA and PB is linearly interpolated so that the data missing point is located on the line connecting points PA and PB on the YZ coordinate plane, and the Z coordinate of the data missing point between points PC and PD is linearly interpolated so that the data missing point is located on the line connecting points PC and PD on the YZ coordinate plane.

[0056] In (S107), the correction unit 223 stores the corrected image data in the corrected image data storage unit 212. Then, the correction unit 223 proceeds to the process of (S108).

[0057] In (S108), the annotation data receiving unit 224 causes the first output device 23 to output an image based on the corrected image data stored in the corrected image data storage unit 212. In this state, an input signal corresponding to an input operation by the user is received from the first input device 24. At this time, the user performs an input operation to specify the type and position of the welding defect in the image output by the first output device 23. In FIG. 10, the type and position of the welding defect are specified by the user's input operation in the image output by the first output device 23.

[0058] When the user's input operation in (S108) is completed, in (S109), the annotation data receiving unit 224 stores the defective welding information identified based on the user's input operation in the annotation data storage unit 213 as annotation data of the defective welding information.

[0059] The AI ​​model generation device 2 repeatedly executes the operations of (S101) to (S109) in the above flowchart to obtain a plurality of data sets of the corrected image data and annotation data of defective welding information for the image data. Thereafter, the AI ​​model generation unit 225 performs machine learning using learning data consisting of the plurality of data sets to generate a defect detection model. Then, the AI ​​model generation unit 225 stores parameters for specifying the generated defect detection model in the parameter storage unit 214. The parameters stored in the parameter storage unit 214 are sent to the defective welding information calculation unit 372 of the welding device 3. The defective welding information calculation unit 372 stores the parameters.

[0060] Thereafter, in the welding device 3, when the 3D sensor 36 captures an image of the welding spot PW, the image processing unit 371 receives the shape data acquired by the 3D sensor 36 and converts it into image data of a welding image including the welding spot PW. Then, the poor welding information calculation unit 372 calculates (predicts) poor welding information based on the image data generated by the image processing unit 371 using a defect detection model specified by the parameters sent from the AI ​​model generation device 2. The poor welding display image generation unit 373 calculates a pixel value of each pixel of the poor welding display image displaying the poor welding information based on the poor welding information calculated by the poor welding information calculation unit 372. Then, the second output device 38 displays the poor welding display image based on the pixel value of each pixel calculated by the poor welding display image generation unit 373.

[0061] Therefore, according to this embodiment 1, when the image data acquired in (S101) contains missing data points, the correction unit 223 corrects the image data and uses the corrected image data for machine learning, thereby suppressing deterioration in the prediction accuracy of the defect detection model caused by the image data used for machine learning containing missing data points.

[0062] In addition, since image data including data missing points can be corrected and the corrected image data can be used as learning data, it is not necessary to repeat photographing with the 3D sensor 36 until learning data without data missing points is obtained. Therefore, the effort required to obtain learning data can be reduced.

[0063] (Embodiment 2) Fig. 11 is a diagram corresponding to Fig. 1 of embodiment 2. In embodiment 2, a defect shape storage unit 215 is further provided in the storage device 21 of the AI ​​model generation device 2. This defect shape storage unit 215 stores 3D data representing predetermined welding defects such as spatters, pits, holes, etc.

[0064] Further, instead of the processes of (S104) to (S106), the correction unit 223 corrects the Z coordinate of a point whose XY coordinate is located in a predetermined correction area including all the data missing points in the pre-correction image data stored in the pre-correction image data storage unit 211 to a value in a state where there is a welding defect. Specifically, the correction unit 223 performs a size adjustment process on the 3D data stored in the defect shape storage unit 215 to enlarge or reduce the welding defect represented by the 3D data to a minimum size that can include all the data missing points specified in (S103) in the XY coordinate plane. Then, the correction unit 223 corrects the pre-correction image data so that a portion located within the correction area in the XY coordinate plane in the 3D image represented by the pre-correction image data stored in the pre-correction image data storage unit 211 is replaced with the welding defect represented by the 3D data after the size adjustment process. The shape and size of the correction area are set to be equal to the shape and size of the welding defect represented by the 3D data after the size adjustment process in the XY coordinate plane. The size of the correction area and the size of the welding defect on the XY coordinate plane after the size adjustment may be set to be larger than the minimum size that can include all of the data missing points.

[0065] For example, the shape of the welding defect represented by the 3D data stored in the defect shape storage unit 215 may be a hemispherical spatter SP bulging outward, as shown in Figures 12 to 15. In the size adjustment process, the diameter of the spatter SP extending in the X-axis direction and the diameter of the spatter SP extending in the Y-axis direction may be enlarged or reduced. In addition, the shape of the welding defect represented by the 3D data stored in the defect shape storage unit 215 may be a hemispherical pit PT dented inward, as shown in Figure 12.

[0066] In the above embodiment 1, the Z coordinate of the data missing point is linearly interpolated based on the coordinates of the points on both outer sides of the missing area in the Y direction. However, the Z coordinate of the data missing point may be linearly interpolated based on the coordinates of the points on both outer sides of the missing area in the X direction.

[0067] In the above first and second embodiments, the functions of the missing point identifying unit 222 and the correction unit 223 and the function of the AI ​​model generating unit 225 are realized by one processor 22, but the functions of the missing point identifying unit 222 and the correction unit 223 and the function of the AI ​​model generating unit 225 may be realized by different processors. In other words, the functions of the processor 22 in the above first and second embodiments may be realized by multiple processors.

[0068] In addition, in the above embodiments 1 and 2, the present invention is applied to generate a defect detection model that predicts welding defect information based on image data, but the present invention can also be applied to detecting objects other than welding defects and generating other models that make specified judgments about images. [Industrial Applicability]

[0069] The training data acquisition device and training data acquisition method disclosed herein can suppress deterioration of the predictive accuracy of the model while reducing the effort required to acquire training data, and are useful as a training data acquisition device and training data acquisition method for acquiring image data to be used in machine learning. [Explanation of symbols]

[0070] 2 AI model generation device (learning data acquisition device) 22 Processor (processing unit) 36 3D Sensor 211 Uncorrected image data storage unit 215 Defective shape memory part

Claims

1. A learning data acquisition device that acquires image data to be used in machine learning for creating a model, an image data storage unit that stores image data of a 3D image acquired by photographing the welding portion with the 3D sensor; Identifying a point whose Z coordinate value satisfies a predetermined condition as a data missing point in the image data stored in the image data storage unit; A processing unit that corrects the Z coordinate of the data missing point in the image data stored in the image data storage unit and acquires the corrected image data as image data to be used for the machine learning.

2. 2. The learning data acquisition device according to claim 1, A learning data acquisition device characterized in that the correction by the processing unit corrects the Z coordinate of the data missing point in the image data stored in the image data memory unit to a value in a state where there is no welding defect.

3. 2. The learning data acquisition device according to claim 1, A learning data acquisition device characterized in that the correction by the processing unit corrects the Z coordinate of the data missing point in the image data stored in the image data memory unit to a value in a state where there is a welding defect.

4. 4. The learning data acquisition device according to claim 3, A defect shape storage unit that stores 3D data representing the predetermined welding defect, The correction by the processing unit is characterized in that the 3D data stored in the defect shape memory unit is subjected to a size adjustment process to enlarge or reduce the welding defect represented by the 3D data, and the image data is corrected so that a portion located within a predetermined correction area including all of the data missing points in the XY coordinate plane in the 3D image represented by the image data stored in the image data memory unit is replaced with the welding defect represented by the 3D data after the size adjustment process.

5. 2. The learning data acquisition device according to claim 1, The correction by the processing unit is characterized in that, in the image data stored in the image data storage unit, the Z coordinate of a point whose XY coordinates are located in a predetermined correction area that includes all of the data missing points in the XY coordinate plane in the 3D image represented by the image data is linearly interpolated based on the Z coordinate of a point whose XY coordinates are located outside the correction area in both the X and Y directions.

6. A learning data acquisition method for acquiring image data to be used in machine learning for creating a model, comprising: Image data of a 3D image acquired by photographing the welding portion with the 3D sensor is stored in an image data storage unit; Identifying a point whose Z coordinate value satisfies a predetermined condition as a data missing point in the image data stored in the image data storage unit; A learning data acquisition method comprising: correcting the Z coordinate of the data missing point in the image data stored in the image data storage unit; and acquiring the corrected image data as image data to be used for the machine learning.

Images