Learning Support System

Abstract

This increases the accuracy of the resulting learning model and improves the efficiency of the process of preparing learning data. [Solution] A learning support system 1 that prepares learning data for generating a learning model that detects detection targets in image data is provided with a data storage 30 that stores at least one set of image data and annotation data indicating the position of the detection target within the image represented by the image data as acquired data, and a data processing unit 10 that acquires status information according to whether the number of detection targets in the images of all image data included in the acquired data has reached a predetermined required number.

Description

[Technical Field]

[0001] The present disclosure relates to a learning support system that prepares learning data for generating a learning model that detects a detection target in image data. [Background technology]

[0002] Patent Document 1 discloses a technology in which, when preparing learning data for generating a learning model that detects detection targets in image data, image data is generated based on shape data measured by a shape measurement unit, and data extension processing is performed on the generated image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 129617 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the number of detection targets included in the learning data used to generate a learning model is too small, the accuracy of the resulting learning model will be low. On the other hand, if data augmentation processing is performed even though sufficient image data has already been obtained before the data augmentation processing, the processing efficiency will be low. Patent Document 1 does not disclose any measures to address these problems.

[0005] The present disclosure has been made in consideration of these points, and its purpose is to increase the accuracy of the obtained learning model and improve the efficiency of the process for preparing learning data. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the present disclosure provides a learning support system that prepares learning data for generating a learning model that detects detection targets in image data, and is characterized by comprising: an original data storage unit that stores at least one set of original data, the image data and annotation data indicating the position of the detection target within the image represented by the image data; and a data processing unit that acquires status information according to whether the number of detection targets in images of all image data included in the original data has reached a predetermined required number.

[0007] This allows a device or a person to refer to the status information and not perform data augmentation processing if the number of detection targets in the original data has reached the required number, thereby improving the efficiency of processing for preparing training data.

[0008] On the other hand, if the number of detection targets in the original data does not reach the required number, the device or a person can refer to the status information and increase the image data by performing data augmentation processing, thereby improving the accuracy of the resulting learning model. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to increase the accuracy of the obtained learning model and improve the efficiency of the process for preparing learning data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic configuration diagram of a learning support system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart illustrating the operation of the data processing unit of the learning support system according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram illustrating a display screen showing the state of data acquired by the data acquisition unit. [Figure 4]FIG. 4 is a table illustrating the types of welding defects, the number of each welding defect in all acquired image data, the number and percentage of each welding defect in acquired image data for learning, and the number and percentage of each welding defect in acquired image data for evaluation. [Figure 5] FIG. 5 is a diagram illustrating an example of a display screen showing information about the training data and evaluation data that are finally prepared. [Figure 6] FIG. 6 is a table illustrating the types of welding defects, the number of each welding defect in the acquired data, the number of each welding defect in the expanded data, and the increased number of each welding defect. [Figure 7] FIG. 7 is a table illustrating examples of types of welding defects, the number of each welding defect in the expanded data, the number and percentage of each welding defect in the learning data included in the expanded data, and the number and percentage of each welding defect in the evaluation data included in the expanded data. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0012] 1 shows a learning support system 1 according to an embodiment of the present disclosure. This learning support system 1 includes a data processing unit 10, a data storage 30 as an original data storage unit, an internal input / output storage area 40, an input device 50, and an output device 60. The internal input / output storage area 40 has first to sixth areas 41 to 46. The input device 50 is configured by, for example, a keyboard 51 and a mouse 52. The output device 60 is configured by, for example, a liquid crystal display.

[0013] The data processing unit 10 includes a data acquisition unit 11, a data preparation unit 12, a learning unit 13, an evaluation unit 14, and an internal storage area 15. The data acquisition unit 11, the data preparation unit 12, the learning unit 13, and the evaluation unit 14 are configured by a GPU (Graphics Processing Unit).

[0014] The data acquisition unit 11 acquires multiple sets of acquired image data of 3D images including the area around the weld and annotation data corresponding to each acquired image data as original data. The welded area is the location where a weld mark is formed, such as a weld bead in laser welding or a dent made by a welding gun in resistance welding. The data acquisition unit 11 receives the acquired image data from an imaging device (not shown), acquires it by processing data received from an imaging device (not shown), or creates it in response to user input. The data acquisition unit 11 acquires a predetermined number of images, including one or more, of acquired image data for learning and acquired image data for evaluation. Hereinafter, "one image data" and "one image file" respectively refer to image data for one image. The annotation data specifies the presence or absence of a welding defect (defect) to be detected in the image represented by the corresponding image data, the size and position of a rectangular area (bounding box) containing the welding defect in the image represented by the corresponding image data, and the type of welding defect within the rectangular area (e.g., perforation, pit, spatter, undercut, protrusion, etc.). The annotation data need only include information indicating the position of the welding defect (detection target) in the image represented by the corresponding image data, and is not limited to the data described in this embodiment. The annotation data is specified by a user's input to the input device 50 while an image based on the acquired image data is output to the output device 60. Specifically, the image is output to the output device 60, and the user specifies a rectangular area containing the welding defect by drawing a frame in the image, and assigns a name of the type of welding defect to the rectangular area. The data acquisition unit 11 stores the acquired image data (image file) for learning and its annotation data in the learning data area 32 of the data storage 30. The data acquisition unit 11 also stores the acquired image data (image file) for evaluation and its annotation data in the evaluation data area 33 of the data storage 30.

[0015] The data preparation unit 12 prepares learning data to be used for learning and evaluation data to be used for evaluation. The learning data consists of learning image data and annotation data for the learning image data. The evaluation data consists of evaluation image data and annotation data for the evaluation image data. The data preparation unit 12 has a data state determination unit 12a, a data extension unit 12b, and a data allocation unit 12c.

[0016] The data state determination unit 12a reads the acquired image data for learning and the acquired image data for evaluation from the data storage 30. Then, the data state determination unit 12a obtains the number of welding defects (number of labels) in all images represented by the acquired image data for learning and evaluation for each type of welding defect, and calculates the total number of welding defects for each type of welding defect. The data state determination unit 12a stores in the internal storage area 15 a defect number total table indicating the calculated total number of welding defects for each type of welding defect.

[0017] The data status determination unit 12a also reads the required number of each welding defect from the operation setting file 31a of the data storage 30 and stores it in the internal storage area 15. The data status determination unit 12a then compares the total number of welding defects shown in the defect total table with the required number for each type of welding defect. The data status determination unit 12a then acquires, as status information for multiple types of welding defects, a data expansion target list indicating types of welding defects whose total number shown in the defect total table does not reach the required number and the difference between the total number of each welding defect and the required number. The data status determination unit 12a stores the acquired data expansion target list in the internal storage area 15. The data status determination unit 12a may also have a function of outputting the acquired data expansion target list to the output device 60.

[0018] The required number of each welding defect is set to the number of labels required to build an AI model. The number of labels required to build an AI model depends on the complexity of the recognition task, the diversity of images, the specific requirements of the AI ​​model being trained, etc. An AI model that detects specific welding defects around weld beads can be built with relatively high performance by learning from image data of multiple images containing a total of 1,000 or more of the welding defects.

[0019] The data extension unit 12b reads the difference from the data extension target list stored in the internal storage area 15. The data extension unit 12b also reads the number of welding defects to be included in an image represented by one image file (one image data) generated by the data extension process from the operation setting file 31a. Then, the data extension unit 12b calculates the number of image files to be generated based on the difference and the number of welding defects to be included in an image represented by one image file generated by the data extension process, and writes the calculated number to the operation setting file 31a.

[0020] The data expansion unit 12b reads the good-quality sample data 34a and the defective-quality sample data 34b from the expansion data area 34 of the data storage 30. The data expansion unit 12b then generates expanded data using the good-quality sample data 34a and the defective-quality sample data 34b. The expanded data includes image data representing an image in which a defective weld included in a defective product image represented by the defective product sample data 34b is pasted onto a good-quality product image represented by the good-quality sample data 34a, and annotation data corresponding to the image data. The good-quality image is an image of a welded area surrounding a weld that does not contain a weld, and the defective product image is an image of a welded area surrounding a weld that contains a weld. The number of expanded data can be increased by pasting the welds onto the good-quality image in multiple positions and directions. The data expansion unit 12b stores the generated expanded data in a second area 42 of the internal input / output memory area 40. The second area 42 is connected to the training data area 32 of the data storage 30.

[0021] The data allocating unit 12c moves all of the acquired image data for evaluation and its annotation data stored in the evaluation data area 33 to the second area 42 of the input / output internal storage area 40, and reads all of the annotation data from the second area 42, one image at a time. The data allocating unit 12c then stores the number of welding defects (number of labels) in each image in the internal storage area 15 for each type of welding defect. The data allocating unit 12c then stores in the internal storage area 15 a defect count total table indicating the total number of welding defects for all images stored in the internal storage area 15 for each type of welding defect. The data allocating unit 12c then reads from the operation setting file 31a a target ratio between the number of detection targets included in the learning data and the number of detection targets included in the evaluation data. This target ratio is set in advance by the user. This target ratio is the target ratio between the number of detection targets in an image represented by the learning image data constituting the learning data and the number of detection targets in an image represented by the evaluation image data constituting the evaluation data. The data allocator 12c moves some of the image data and annotation data stored in the second area 42 to the third area 43 so that, for each type of welding defect, the ratio between the number of detection targets included in the image data in the second area 42 and the number of detection targets included in the image data in the third area 43 approaches the target ratio. After this data movement, the image data and annotation data stored in the second area 42 become learning data, and the image data and annotation data stored in the third area 43 become evaluation data. The target ratio between the number of detection targets included in the learning data and the number of detection targets included in the evaluation data is set to, for example, 7:3.

[0022] The learning unit 13 reads the image data and the corresponding annotation data sorted by the data sorting unit 12c to the second area 42. That is, the learning unit 13 reads the learning data. Then, the learning unit 13 generates a learning model based on the read image data and annotation data. A known inference engine, for example, YOLO, an object detection algorithm, is implemented in the learning unit 13. The learning unit 13 stores the generated learning model in the data storage 30 as a learning model file 35a. The learning model detects welding defects, which are multiple types of detection targets, in the image data. The input data of the learning model is the image data. The output data of the learning model identifies the presence or absence of welding defects in an image based on the image data, the size and position of a rectangular area (bounding box) containing the welding defect in the image based on the image data, and the type of welding defect (e.g., perforation, pit, spatter, undercut, protrusion, etc.) within the rectangular area.

[0023] The evaluation unit 14 reads the image data and the annotation data thereof sorted by the data sorting unit 12c to the third area 43. That is, the evaluation unit 14 reads evaluation data. The evaluation data is used by the evaluation unit 14 to evaluate the learning model generated by the learning unit 13. Then, the evaluation unit 14 calculates the total number of correct detections, the total number of false detections, the total number of undetected detections, and the accuracy of the detection position (Intersection over Union) based on the output data when the read image data is input to the learning model generated by the learning unit 13 and the read annotation data. Here, the total number of correct detections is the number of welding defects identified by the output data of the trained model that are included in the welding defects identified by the annotation data. Furthermore, the total number of false detections is the number of welding defects identified by the output data of the trained model that are not included in the welding defects identified by the annotation data. Furthermore, the total number of undetected detections is the number of welding defects identified by the annotation data that are not included in the welding defects identified by the output data of the trained model. The accuracy of the detected position is calculated by dividing the area of ​​a rectangular region that is both identified by the annotation data and the output data of the trained model by the area of ​​the region included in at least one of the rectangular regions identified by the annotation data and the output data of the trained model. The evaluation unit 14 writes the calculated total number of correct detections, total number of false detections, total number of undetected detections, and accuracy of the detected position to the evaluation result file 36a of the data storage 30.

[0024] Furthermore, the evaluation unit 14 stores the calculated total number of correct detections, total number of incorrect detections, total number of undetected detections, and accuracy of the detection positions in the internal storage area 15. Next, the evaluation unit 14 calculates a correct detection rate (Precision) and a detection rate (Recall) based on the stored total number of correct detections, total number of incorrect detections, and total number of undetected detections.

[0025] The correct detection rate, detection rate, and accuracy of the detected position are expressed by the following formulas (1) to (3).

[0026] Here, the correct detection rate is Pr, the detection rate is Re, the accuracy of the detected position is IoU, the total number of correct detections is TP, the total number of false detections is FP, the total number of undetected detections is FN, the area of ​​the area included in at least one of the rectangular area identified by the annotation data and the rectangular area identified by the output data of the trained model is S1, and the area of ​​the area that is a rectangular area identified by the annotation data and also a rectangular area identified by the output data of the trained model is S2.

[0027] Pr = TP / (TP + FP) (1) Re=TP / (TP+FN) (2) IoU=S2 / S1 (3) The evaluation unit 14 also reads a weight setting pattern from the operation setting file 31a of the data storage 30 and stores it in the first area 41 of the input / output internal storage area 40. Then, the evaluation unit 14 calculates a performance score, which is the sum of products of three types of evaluation indexes and weights for the trained model, for the weight setting pattern stored in the first area 41. The three types of evaluation indexes are the correct detection rate, the detection rate, and the accuracy of the detected position. The evaluation unit 14 also writes the calculated performance score to the evaluation result file 36a of the data storage 30. Here, if the performance score for the trained model is Score, the weight of the correct detection rate is w_p, the weight of the detection rate is w_r, and the weight of the accuracy of the detected position is w_iou, the performance score is expressed by the following equation (4).

[0028] Score=Pr*w_p+Re*w_r+IoU*w_iou ···(4) The operation of the data processing unit 10 will be described below with reference to the flowchart of FIG.

[0029] First, in S101, the data acquisition unit 11 acquires a predetermined number of images, one or more, of acquired image data for learning and acquired image data for evaluation. The data acquisition unit 11 also acquires annotation data corresponding to each acquired image data. The data acquisition unit 11 then stores the acquired image data for learning and its annotation data in the learning data area 32 of the data storage (original data storage unit) 30. The data acquisition unit 11 also stores the acquired image data for evaluation and its annotation data in the evaluation data area 33 of the data storage (original data storage unit) 30. The acquired image data for learning and evaluation stored in the data storage 30 constitute all image data included in the original data.

[0030] Next, in S102, the data state determination unit 12a reads all acquired image data for learning and evaluation from the data storage 30. Then, the data state determination unit 12a obtains the number of welding defects (number of labels) for each type of welding defect in all images represented by the read acquired image data, and calculates the total number of welding defects for each type of welding defect. The data state determination unit 12a stores in the internal storage area 15 a defect number total table indicating the calculated total number of welding defects for each type of welding defect.

[0031] Next, in S103, the data state determination unit 12a reads the required number of each welding defect from the operation setting file 31a of the data storage 30 and stores it in the internal storage area 15. Then, the data state determination unit 12a compares the total number of welding defects shown in the defect number total table with the required number for each type of welding defect, and identifies the type of welding defect whose total number shown in the defect number total table is less than the required number. In other words, the data state determination unit 12a identifies the type of welding defect to be subjected to data expansion.

[0032] Next, in S104, the data status determination unit 12a stores a data expansion target list indicating the types of welding defects identified in S103 that are to be subject to data expansion and the difference between the total number of each welding defect that is to be subject to data expansion and the required number in the internal memory area 15 as status information for multiple types of welding defects.

[0033] 3 illustrates a display screen showing the state (initial state) of the acquired data acquired by the data acquisition unit 11. The data state determination unit 12a may have a function to output such a display screen to the output device 60.

[0034] In the example of FIG. 3 , the data acquisition unit 11 acquires acquired image data for 34 images. The required number of each welding defect is set to 100. The total number (number of labels) of holes, undercuts, pits, spatters, and projections in all 34 images shown by the acquired image data acquired by the data acquisition unit 11 is 24, 1, 89, 682, and 4, respectively. The differences (required_labels) between the total number of holes, undercuts, pits, spatters, and projections and the required number are 76, 99, 11, 0, and 96, respectively. The acquired image data for learning is 31 of the 34 images, and the acquired image data for evaluation is 3 of the 34 images. The total number (number of labels) of holes, undercuts, pits, spatters, and projections in all 31 images shown by the acquired image data for training was, respectively, 24, 1, 88, 661, and 1. The total number (number of labels) of holes, undercuts, pits, spatters, and projections in all three images shown by the acquired image data for evaluation was, respectively, 0, 0, 1, 21, and 3.

[0035] In the example of Fig. 3, the ratio of the number of holes in the acquired image data for learning to the number of holes in all the acquired image data is 100%, as shown in Fig. 4. Also, the ratio of the number of holes in the acquired image data for evaluation to the number of holes in all the acquired image data is 0%.

[0036] The ratio of the number of undercuts in the acquired image data for learning to the number of undercuts in all acquired image data is 100%, and the ratio of the number of undercuts in the acquired image data for evaluation to the number of undercuts in all acquired image data is 0%.

[0037] The ratio of the number of pits in the acquired image data for learning to the number of pits in all the acquired image data is 98.88%, and the ratio of the number of pits in the acquired image data for evaluation to the number of pits in all the acquired image data is 1.12%.

[0038] The ratio of the number of spatters in the acquired image data for learning to the number of spatters in all acquired image data is 96.92%, and the ratio of the number of spatters in the acquired image data for evaluation to the number of spatters in all acquired image data is 3.08%.

[0039] The ratio of the number of projections in the acquired image data for learning to the number of projections in all acquired image data is 25%, and the ratio of the number of projections in the acquired image data for evaluation to the number of projections in all acquired image data is 75%.

[0040] Next, in S105, the data extension unit 12b reads the difference from the data extension target list stored in the internal storage area 15. Furthermore, the data extension unit 12b reads the number of welding defects to be included in an image represented by one image file (one image data) generated by the data extension process from the operation setting file 31a. Then, the data extension unit 12b calculates the number of image files to be generated based on the difference and the number of welding defects to be included in an image represented by one image file generated by the data extension process, and writes the number of image files to be generated to the operation setting file 31a.

[0041] Next, in S106, the data expansion unit 12b reads the good product sample data 34a and the defective product sample data 34b from the expansion data area 34 of the data storage 30. Then, the data expansion unit 12b generates expanded data using the good product sample data 34a and the defective product sample data 34b. The expanded data includes image data representing an image in which the good product image represented by the good product sample data 34a is pasted with the welding defect included in the defective product image represented by the defective product sample data 34b, and annotation data corresponding to the image data. The number of image files (image data) generated here, i.e., the number of image files included in the expanded data, is the number of image files calculated in S105.

[0042] Next, in S107, the data allocating unit 12c allocates the acquired data (original data) acquired in S101 and the extended data generated in S106 into learning data and evaluation data. This allocation is performed so that the ratio of learning data to evaluation data approaches the target ratio for each type of welding defect. At this time, the data allocating unit 12c temporarily moves the acquired evaluation data stored in the evaluation data area 33 to the second area 42 of the input / output internal storage area 40. Thereafter, the data allocating unit 12c moves the data to be allocated to evaluation data to the third area 43.

[0043] Fig. 5 illustrates a display screen showing information about the training data and evaluation data that are finally prepared when the acquired data is in the state illustrated in Fig. 3. The data sorting unit 12c may have a function to output such a display screen to the output device 60.

[0044] 5, the combined extended data of the acquired data and the extended data is equivalent to 156 images. In other words, the extended data generated in S106 is equivalent to 122 images.

[0045] The total number (number of labels) of holes, undercuts, pits, spatters, and projections in all 156 images shown by the expanded data is 139, 101, 111, 682, and 100, respectively.

[0046] As shown in Figure 6, by expanding the acquired data to the expanded data, the total number (number of labels) of holes, undercuts, pits, spatters, and projections increases by 115, 100, 22, 0, and 96, respectively.

[0047] In the acquired data, the average number of each weld defect was 160, with a standard deviation of 293.95. In the expanded data, the average number of each weld defect was 226.6, with a standard deviation of 255.06.

[0048] 7, the expanded data for 156 images is divided into training data for 106 images and evaluation data for 50 images. The training data for 106 images includes acquired data for 22 images acquired in S101. The evaluation data for 50 images includes acquired data for 12 images acquired in S101. The total numbers (number of labels) of holes, undercuts, pits, spatters, and projections in all 106 images represented by the training data are 99, 72, 87, 477, and 70, respectively. The total number (number of labels) of holes, undercuts, pits, spatters, and projections in all 50 images shown in the evaluation data is 40, 29, 24, 205, and 30, respectively.

[0049] 7, the ratio of the number of holes in the training data to the number of holes in all the expanded data is 71.22%, and the ratio of the number of holes in the evaluation data to the number of holes in all the expanded data is 28.88%.

[0050] The ratio of the number of undercuts in the training data to the number of undercuts in all the expanded data is 71.29%, and the ratio of the number of undercuts in the evaluation data to the number of undercuts in all the expanded data is 28.7%.

[0051] The ratio of the number of pits in the training data to the number of pits in all the expanded data is 78.38%, and the ratio of the number of pits in the evaluation data to the number of pits in all the expanded data is 21.62%.

[0052] The ratio of the number of spatters in the training data to the number of spatters in all the expanded data is 69.99%, and the ratio of the number of spatters in the evaluation data to the number of spatters in all the expanded data is 30.01%.

[0053] The ratio of the number of projections in the training data to the number of projections in all the expanded data is 70%, and the ratio of the number of projections in the evaluation data to the number of projections in all the expanded data is 30%.

[0054] 5, of the 106 images represented by the training data, the total numbers (number of labels) of holes, undercuts, pits, spatters, and projections in 22 images acquired in S101 are 17, 0, 75, 477, and 0, respectively. Of the 50 images represented by the evaluation data, the total numbers (number of labels) of holes, undercuts, pits, spatters, and projections in 12 images acquired in S101 are 7, 1, 14, 205, and 4, respectively.

[0055] Next, in S108, the learning unit 13 reads the learning data allocated to the second area 42 by the data allocation unit 12c. Then, the learning unit 13 generates a learning model based on the read image data and annotation data.

[0056] Next, in S109, the evaluation unit 14 reads the evaluation data allocated to the third area 43 by the data allocation unit 12c. The evaluation unit 14 then calculates the total number of correct detections, the total number of false detections, the total number of undetected results, and the accuracy of the detection positions (Intersection over Union) based on the output data when the read image data is input to the learning model generated by the learning unit 13 and the read annotation data. The evaluation unit 14 then writes the total number of correct detections, the total number of false detections, the total number of undetected results, and the accuracy of the detection positions to the evaluation result file 36a of the data storage 30. The evaluation unit 14 then calculates the correct detection rate (Precision) and the detection rate (Recall) based on the saved total number of correct detections, the total number of false detections, and the total number of undetected results. The evaluation unit 14 then reads a weight setting pattern from the operation setting file 31a of the data storage 30 and saves it in the first area 41 of the input / output internal storage area 40. Then, the evaluation unit 14 calculates the performance score, which is the sum of the products of the three types of evaluation indexes and the weights for the learning model, for the weight setting pattern stored in the first area 41.

[0057] As described above, according to this embodiment, when the number of welding defects in the acquired data reaches the required number, the data extension unit 12b of the data processing unit 10 does not perform the data extension process, thereby improving the efficiency of the process of preparing learning data and evaluation data.

[0058] On the other hand, when the number of welding defects in the acquired data does not reach the required number, the data expansion unit 12b of the data processing unit 10 can automatically increase the image data by performing data expansion processing by referring to the data expansion target list, thereby improving the accuracy of the obtained learning model.

[0059] Furthermore, the data extension unit 12b of the data processing unit 10 generates the extended data in a number corresponding to the difference between the number of welding defects in the acquired data and the required number, thereby reducing unnecessary data extension processing and improving processing efficiency. Furthermore, the accuracy of the obtained learning model can be reliably improved.

[0060] In the past, the task of preparing a data set (a set of image data and annotation data) containing a balanced number of types of welding defects as learning data and evaluation data, and the task of appropriately allocating the expanded data into learning data and evaluation data, required a user to have advanced technical knowledge. Furthermore, these tasks required a great deal of labor and time. In this embodiment, the learning support system 1 automatically performs these tasks, thereby reducing the technical knowledge, labor, and time required of the user.

[0061] In this embodiment, the data extension unit 12b of the data processing unit 10 determines whether to perform the data extension process by referring to the data extension target list. However, a person may recognize the data extension target list and perform a predetermined input operation on the data processing device, and the data processing device may determine whether to perform the data extension process in accordance with the input operation.

[0062] In this embodiment, the status information acquired by the data status determination unit 12a is a data expansion target list. However, the status information may be other information as long as it is information indicating whether the total number of multiple types of welding defects in images represented by all acquired image data acquired by the data acquisition unit 11 reaches the required number. For example, the status information may be information identifying only the types of welding defects for which the total number of welding defects in images represented by all acquired image data acquired by the data acquisition unit 11 does not reach the required number, and may not include information indicating the difference between the total number of welding defects and the required number. Note that in this embodiment, the required number for multiple types of welding defects is set to a common number of 100, but the required numbers for multiple types of welding defects may be different numbers. In addition, when generating a learning model for detecting one type of welding defect, the status information is information indicating whether the total number of welding defects in images represented by all acquired image data acquired by the data acquisition unit 11 reaches the required number.

[0063] Furthermore, in this embodiment, the learning support system 1 is provided with a learning unit 13 and an evaluation unit 14. However, the learning support system 1 may not be provided with the learning unit 13 and the evaluation unit 14, and the functions of the learning unit 13 and the evaluation unit 14 may be performed by another system or device.

[0064] In this embodiment, the present invention is applied to the case where training data and evaluation data are prepared, but the present invention can also be applied to the case where only training data is prepared. In this case, the data acquisition unit 11 only needs to acquire at least one set of acquired image data and corresponding annotation data as original data.

[0065] Furthermore, in this embodiment, the detection target is a defective weld, but the present invention can also be applied to cases where the detection target is something other than a defective weld, such as a person.

[0066] Furthermore, in this embodiment, the target area including the welding defect is a rectangular area, but it may be an area of ​​a shape other than a rectangle.

[0067] In this embodiment, the multiple evaluation indexes used to calculate the performance score in S109 are three types of evaluation indexes: the correct detection rate, the detection rate, and the accuracy of the detected position. However, two or four or more types of evaluation indexes may be used as long as there are multiple types of evaluation indexes. Furthermore, the multiple evaluation indexes used to calculate the performance score in S109 may include only one or two of the correct detection rate, the detection rate, and the accuracy of the detected position. [Industrial Applicability]

[0068] The learning support system of the present disclosure can increase the accuracy of the obtained learning model and improve the efficiency of the process of preparing learning data, and is therefore useful as a learning support system that prepares learning data for generating a learning model that detects detection targets in image data. [Explanation of symbols]

[0069] 1. Learning support system 10 Data Processing Unit 30 Data storage (original data storage section) 60 Output Device

Claims

1. A learning support system that prepares learning data for generating a learning model that detects a detection target in image data, an original data storage unit that stores, as original data, at least one set of the image data and annotation data that indicates the position of the detection target within the image represented by the image data; A learning support system characterized by comprising a data processing unit that acquires status information according to whether the number of detection targets in images of all image data included in the original data has reached a predetermined required number.

2. The learning support system according to claim 1, The learning model detects a plurality of types of detection targets in image data, A learning support system characterized in that the status information corresponds to whether or not the number of each of the multiple types of detection targets in all image data contained in the original data has reached a predetermined required number.

3. The learning support system according to claim 2, A learning support system characterized in that the status information is information that identifies the type of detection object in which the number of detection objects in images represented by all image data included in the original data does not reach the specified required number.

4. The learning support system according to claim 1, the status information indicates a difference between the number of detection targets and the required number, The learning support system is characterized in that the data processing unit generates the image data and the annotation data as extended data in a number of pairs corresponding to the difference.

5. The learning support system according to claim 4, The data processing unit further prepares evaluation data to be used for evaluating the learning model; A learning support system characterized in that the original data and the extended data are divided into the learning data and the evaluation data.

6. 6. The learning support system according to claim 5, A learning support system characterized in that the data processing unit allocates the original data based on a target ratio between the number of detection targets in an image represented by image data included in the learning data and the number of detection targets in an image represented by image data included in the evaluation data.

7. The learning support system according to claim 1, The learning support system further comprises an output device for outputting the state information.

Images