Method for generating a re-learning model, device for generating a re-learning model, and program for generating a re-learning model

Abstract

To provide a generation method for a re-learning method that reduces burden of re-learning a learning model in machine learning.SOLUTION: A generation method for a re-learning model for an object detection model in machine learning performed by an information processor comprises the steps of: outputting, using a learned model, a prediction result including a label of a candidate object and text data of a position coordinate and a confidence level from new image data; creating, on the basis of the confidence level and a feature of the candidate object included in the new image data, a re-learning dataset; and performing re-learning using the re-learning dataset.SELECTED DRAWING: Figure 2A

Description

[Technical Field]

[0001] The present disclosure relates to the field of retraining a trained object detection model generated using machine learning such as a neural network, and more particularly to automatically generating a training dataset for use in retraining. [Background technology]

[0002] The accuracy of learning models generated using machine learning techniques such as neural networks depends on the data used for training. Therefore, if the initial training data is limited or biased, the model's accuracy will decrease. For example, when generating a learning model to classify defective products from images, there may be cases where sufficient images of defective products are not available for training because defective products are rarely produced. Even if there is sufficient training data, the model's vulnerability to data that is extrapolated from irregular cases becomes apparent. For example, images of defective products caused by chronic issues may be sufficient as training data, but defective products caused by irregular cases such as sudden malfunctions may not be identified. Therefore, to maintain the ongoing accuracy of a learning model, "relearning" is required, in which training data is continually added and training is performed.

[0003] Conventionally, in the task of generating training data when creating a learning model, a method has existed in which pseudo labels that are candidates for the correct label are generated to determine the correct label to be assigned to an object in an image, and a judgment is made from the pseudo labels using a combination of human visual inspection and calculations (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-89491

[0005] Japanese Patent Laid-Open Publication No. 2006-2030639 (JP-A-2006-2030639) introduces a method for reducing the burden of annotation work, which is one part of the work of generating a learning model, by adding metadata to training data. While this is effective in cases where the training data is predetermined, since the work of generating a learning model must primarily begin with the selection of training data, in cases such as re-training, this work of selecting training data becomes a significant burden. Specifically, in re-training, training data is extracted from images that the existing learning model was unable to judge. Therefore, there is a problem in that the data for re-training must be extracted from a large amount of image data that has already been judged by the existing model. Furthermore, in the case of assigning only correct labels as metadata to image data for learning, such as an image classifier, among other learning models, it is possible to reduce the workload using a method such as that described in Patent Document 1. Summary of the Invention [Problem to be solved by the invention]

[0006] However, when learning data that includes location information in addition to the correct labels as metadata for object detection and region detection is required, the location information must be added manually. In particular, when performing region extraction, the number of coordinate data points for the location information to be added is extremely large, which creates a significant workload. An object of the present disclosure is to provide an efficient re-learning method that solves the above-mentioned problems. [Means for solving the problem]

[0007] A method for generating a re-learning model executed by an information processing device according to an embodiment of the present disclosure includes: A method for generating a retraining model for an object detection model in machine learning, comprising: a learning step of generating a trained model using an initial training dataset including a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; a prediction step of outputting a prediction result including text data of a label, position coordinates, and confidence level of a candidate object from new image data using the generated trained model; a re-learning data generation step of creating a re-learning data set from the prediction result based on the confidence level and feature amounts of the candidate object included in the new image data; a retraining step of generating a retrained model using the retrained dataset; It has.

[0008] An apparatus for generating a re-learning model according to an embodiment of the present disclosure includes: A device for generating a re-learning model for an object detection model in machine learning, comprising: a learning unit that generates a trained model using an initial training dataset that includes a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; a prediction unit that uses the generated trained model to output a prediction result including text data of a label, position coordinates, and confidence level of a candidate object from new image data; a re-learning data generation unit that creates a re-learning data set from the prediction result based on the confidence level and feature amounts of the candidate object included in the new image data; a re-training unit that generates a re-trained model using the re-training dataset; Equipped with.

[0009] A re-learning model generation program according to an embodiment of the present disclosure includes: A program for generating a retraining model for an object detection model in machine learning, comprising: generating a trained model using an initial training dataset including a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; outputting a prediction result including a label of a candidate object and text data of its position coordinates and confidence level from new image data using the generated trained model; creating a re-learning dataset from the prediction result based on the confidence level and the feature amount of the candidate object included in the new image data; generating a retrained model using the retrained dataset; This is a program that executes the above. [Effects of the Invention]

[0010] According to an embodiment of the present disclosure, it is possible to significantly reduce the burden of relearning work for learning models, which has conventionally required a great deal of manpower. Furthermore, since learning data can be continuously and automatically relearned, it is possible to reduce the amount of learning data when generating an initial model, and it is possible to reduce the number of steps from model development to implementation. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a re-learning model generation device according to an embodiment of the present disclosure. [Figure 2A] FIG. 1 is a diagram illustrating an overview of a method and apparatus for generating a re-training model according to an embodiment of the present disclosure. [Figure 2B] FIG. 1 is a diagram illustrating an overview of a method and apparatus for generating a re-training model according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart illustrating a procedure of a re-learning model generation method according to an embodiment of the present disclosure. [Figure 4A] FIG. 10 is a diagram illustrating an overview of a detection result according to an embodiment of the present disclosure. [Figure 4B] FIG. 10 is a diagram illustrating an overview of a detection result according to an embodiment of the present disclosure. [Figure 4C] FIG. 10 is a diagram illustrating an overview of a detection result according to an embodiment of the present disclosure. [Figure 5A] FIG. 10 is a diagram illustrating an example of determining re-learning data according to an embodiment of the present disclosure. [Figure 5B] FIG. 10 is a diagram illustrating an example of determining re-learning data according to an embodiment of the present disclosure. [Figure 6A]FIG. 10 is a diagram illustrating an example of a summary of a detection result according to an embodiment of the present disclosure. [Figure 6B] FIG. 10 is a diagram illustrating an example of a summary of a detection result according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a diagram illustrating examples of various machine learning parameters and performance evaluation indicators according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, a method for generating a relearning model and a device for generating a relearning model according to an embodiment of the present disclosure will be described with reference to the drawings.

[0013] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0014] (Overview of the re-learning model generation device 20) The configuration of an information processing device 20 (hereinafter referred to as the generating device 20) for generating a re-learning model according to this embodiment will be described below. Fig. 1 is a configuration diagram of the generating device 20 for a re-learning model according to this embodiment.

[0015] The relearning model generation device 20 includes a learning unit 21, a prediction unit 22, a prediction result processing unit 23, a relearning data candidate generation unit 24, a relearning data selection unit 25, a dataset creation unit 26, a relearning data generation unit 27, a relearning unit 28, a learning parameter optimization unit 29, a performance evaluation unit 30, and a judgment unit 31.

[0016] The learning unit 21 performs machine learning on an initial learning dataset 3 including initial image data 1, correct labels, and text data 2 of position coordinates using an algorithm for object detection, and generates a learned model 4.

[0017] The prediction unit 22 uses the trained model 4 generated by the learning unit 21 to infer new image data 5, and obtains an output result for the candidate object in the image as a prediction result 6. The prediction result 6 may include the output result of the model including the label, position coordinate data, and confidence level of the candidate object.

[0018] The prediction result processing unit 23 uses an arbitrarily determined first threshold related to the confidence level of the obtained prediction results 6, and adopts prediction results with a confidence level equal to or greater than the first threshold as predictions in which an object has been detected. On the other hand, the prediction result processing unit 23 rejects prediction results with a confidence level less than the first threshold as predictions in which an object has not been detected. That is, even if a candidate object is detected as a result of a prediction using the trained model 4 for new image data 5, prediction results 6 with a confidence level less than the first threshold are treated as rejects, meaning that no object has been detected. On the other hand, prediction results 6 with a confidence level equal to or greater than the first threshold are treated as rejects, meaning that an object has been detected. The output of the prediction result processing unit 23 is the result of object detection by machine learning.

[0019] Next, the re-learning technique will be described. The re-learning data candidate generation unit 24, among the prediction results 6, those with a confidence level less than the second threshold, combines them with the corresponding new image data 5 to form a re-learning candidate dataset 7 (see FIG. 2A ). The prediction results 6 determined to be adopted by the prediction result processing unit 23 include prediction results 6 with high confidence and prediction results 6 with low confidence in the object detection results. The re-learning data candidate generation unit 24 extracts the prediction results 6 with low confidence as the re-learning candidate dataset 7. In other words, the re-learning data candidate generation unit 24 extracts the prediction results 6 with a confidence level less than the second threshold as the re-learning candidate dataset 7. Here, the re-learning candidate dataset 7 may include prediction results 6 determined to be rejected by the prediction result processing unit 23.

[0020] The re-learning candidate dataset 7 contains a mixture of a group of objects 8 that should have been detected and a group of objects 9 that should not have been detected. That is, the low-confidence prediction results 6 include prediction results 6 that were correctly judged to be adopted and prediction results 6 that should not have been adopted. Furthermore, the rejected prediction results 6 may also include prediction results 6 that were correctly judged to be rejected and prediction results 6 that should have been adopted. Therefore, the re-learning data selection unit 25 sorts these and selects a re-learning dataset 10 consisting of the group of objects 8 that should have been detected.

[0021] The classification of data for re-learning is determined based on predetermined feature quantities related to candidate objects. Specifically, the determination based on predetermined feature quantities is performed by setting a third threshold value such as brightness value, RGB value, HSV value, etc. However, it is effective to perform the classification determination after removing noise by binarizing the image data or grayscale processing.

[0022] The dataset creation unit 26 combines the re-training dataset 10 with an existing training dataset to create a new training dataset 11. The new training dataset 11 is used by a re-training unit 28 to generate a re-trained model 12. The training parameters 13 used in this process may be the same as the training parameters used to generate the trained model 4, but preferably, the training parameters optimized by a training parameter optimization unit 29 are used.

[0023] The relearning data candidate generation unit 24, the relearning data selection unit 25, and the dataset creation unit 26 are components for creating a dataset for relearning, and are collectively referred to as a relearning data generation unit 27. The dataset creation unit 26 is optional, and when a new training dataset 11 is not created, the relearning data candidate generation unit 24 outputs the relearning dataset 10 to the relearning unit 28.

[0024] The learning parameter optimization unit 29 generates various parameter values ​​13a, 13b, etc. that serve as conditions during learning, generates a learning model using those parameters, and uses each performance evaluation index 14 of the learning model to adopt the parameters that result in the best performance.

[0025] The various parameters used include the learning rate, the hold-out ratio when dividing the entire data into training data and validation data in the hold-out method, momentum, weight decay, etc. Performance evaluation indices 14 used include precision, recall, IOU, etc.

[0026] The performance evaluation unit 30 evaluates the performance of the re-trained model 12 and the existing trained model 4. The judgment unit 31 uses the performance evaluation index 14 of the re-trained model 12 and the existing trained model 4 provided by the performance evaluation unit 30 to make a judgment on switching the trained models.

[0027] According to an embodiment of the present disclosure, the device for generating a re-training model includes a re-training data generation unit that generates a candidate re-training dataset together with new image data corresponding to a prediction result in which the confidence level is less than a second threshold and the feature quantities of the candidate object satisfy a predetermined standard, and creates a re-training dataset from the candidate re-training dataset, thereby significantly reducing the burden of re-training a learning model, which previously required a great deal of manpower. Furthermore, because the device can continuously and automatically re-train the learning data, it is possible to reduce the amount of learning data when generating an initial model, thereby reducing the man-hours from model development to implementation.

[0028] (Configuration of a method and device for generating a re-learning model) The method and device for generating a re-learning model according to this embodiment will be described in detail with reference to FIGS. 2A and 2B.

[0029] As shown in FIG. 2A , in this embodiment, an initial training dataset 3 is first prepared, including initial image data 1 for initial training and text data 2 describing correct labels and position coordinates as metadata for objects (objects) in the image. The image data may be color, grayscale, or monochrome, and preprocessing such as contrast adjustment may be performed as necessary. The resolution and data format are optional. The text data may be in a data format that describes information corresponding to the algorithm to be used using a markup language such as XML. The position coordinate data to be described may be written in a format that describes the coordinates of four points in the case of a rectangular box, or in a format that describes the coordinates of the origin and the height and width of the box. In the case of an area, the coordinates of the vertices of the area are specified. The origin may be determined arbitrarily. An image may contain multiple objects, but one piece of text data is prepared for each piece of image data.

[0030] 4A to 4C show examples of image data and text data to be used. Two objects (object ob1 and object ob2) exist in the image, and the text data contains the label and position coordinate data for each object as region data in XML format.

[0031] In this way, the learning unit 21 learns the initial learning dataset 3, which includes multiple sets of initial image data 1 and the corresponding text data 2, using an object detection algorithm. The hardware constituting the learning unit 21 can be determined arbitrarily based on the number of learning data sets to be used, the capacity of the initial image data 1, the type of learning algorithm, etc., but preferably uses hardware equipped with a core capable of performing complex calculations in parallel. The algorithm used here may be a commercially available learning algorithm for image recognition or an open source algorithm published on the Internet. For object detection, it is preferable to use Faster R-CNN or YOLO, and for region detection, it is preferable to use Detectron.

[0032] The trained model 4 generated by the learning unit 21 is installed in the hardware that constitutes the prediction unit 22. The hardware that constitutes the prediction unit 22 may also be the hardware that constitutes the learning unit 21. New image data 5 that is different from the initial image data 1 is inferred by the trained model 4 of the prediction unit 22, and the output result for the object in the image is obtained as a prediction result 6. The prediction result 6 only needs to include the output result of the model, including the object's label, position coordinate data, and confidence level.

[0033] Of the obtained prediction results 6, a prediction result processor 23 uses prediction results with a certainty level equal to or greater than an arbitrarily determined first threshold as an output result indicating that an object has been detected, and uses prediction results with a certainty level less than the first threshold as an output result indicating that no object has been detected, and rejects the results. The output of this prediction result processor 23 becomes the result of object detection by machine learning.

[0034] Next, the relearning data candidate generation unit 24 sets the prediction results whose confidence level is less than the second threshold together with the corresponding image data as the relearning candidate dataset 7 for relearning. Note that the prediction results whose confidence level is less than the second threshold and equal to or greater than the first threshold may also be set together with the corresponding image data as the relearning candidate dataset 7. Candidate image data in which candidate positions of objects generated from text data of the position coordinates of the objects in the prediction results are superimposed may be used as the image data of the relearning candidate dataset 7.

[0035] The re-learning candidate dataset 7 contains a mixture of a group of objects 8 that should be detected and a group of objects 9 that should not be detected, so the re-learning data selection unit 25 sorts these and selects a re-learning dataset 10 consisting of the group of objects 8 that should be detected.

[0036] The classification of data for relearning is determined based on predetermined feature values ​​of candidate objects included in new image data of the candidate relearning dataset. The predetermined feature values ​​include brightness, RGB, and HSV values. For example, when brightness is used as the predetermined feature value, the classification of data for relearning is determined based on whether the brightness is greater than or less than a third threshold. In this way, the classification of data for relearning is determined based on whether a third threshold value, such as brightness, RGB, or HSV, is set. Furthermore, image data can be classified after noise removal using binarization, grayscale processing, or other methods. When classifying data, classification may be performed visually if the criteria for determining the object are clear to the human eye, such as object labels (e.g., dog, car, text, etc.).

[0037] For example, in the re-learning data selection unit 25 of a model for detecting scratches on the surface of steel, when images such as those shown in Figures 5A and 5B are proposed as candidate re-learning data sets 7, if brightly displayed scratch areas are found in the candidate data after image processing, the image is adopted as re-learning data. Specifically, after binarizing the image using a brightness threshold of 128, image (1) in which bright areas with a brightness of 255 are found is adopted as re-learning data, and image (2) in which only dark areas with a brightness of 0 are found is not adopted as re-learning data.

[0038] Note that among the rejected prediction results, there is image data in which no objects exist. Such image data in which no objects exist may also be included in the re-learning candidate dataset 7 as training data for machine learning.

[0039] At this time, if the labels or position coordinates of the relearning dataset 10 need to be corrected, the relearning data selection unit 25 has a function to manually perform the corrections. In other words, when generating the relearning data, the text data of the labels or position coordinates of the prediction results may be corrected. FIGS. 6A and 6B show examples of output text data and aggregation results. The text data contains the labels for each detected object, the coordinates of each vertex of the area, and the confidence level. These are aggregated and used to determine the candidates for relearning based on a threshold value. The results are further sorted into a group 8 of objects that should be detected and a group 9 of objects that should not be detected.

[0040] Next, as shown in FIG. 2B , a dataset creation unit 26 combines the re-training dataset 10 with an existing training dataset to create a new training dataset 11, and a re-training unit 28 generates a re-training model 12. The training at this time may be performed using the same hardware as the training unit 21, but the same training algorithm as that used for the training unit 21 is used. The training parameters 13 at this time may be the same as those used when generating the trained model 4, but preferably those optimized by a learning parameter optimization unit 29 are used.

[0041] The learning parameter optimization unit 29 can use a method of randomly generating various parameter values ​​13a, 13b, etc. that serve as conditions during learning, generating a learning model using those parameters, and adopting the parameters that provide the best performance, or a method of generating various parameter values ​​13a, 13b, etc. within a predetermined range and adopting the parameters that provide the best performance.

[0042] Specifically, the superiority or inferiority of a parameter may be determined by the total value of each performance evaluation index 14. A parameter with a high performance evaluation index suited to the environment in which the model is used may be adopted, such as recall if priority is given to the overdetection rate, precision if priority is given to the overlooked rate, or IOU if priority is given to the positional accuracy of the detection results.

[0043] Figure 7 shows examples of generated parameters and the results of evaluating model performance when training using each parameter. The various parameters are at least four for machine learning models using neural networks: the learning rate, the hold-out ratio (the ratio of training data to validation data in the hold-out method), momentum, and weight decay. Each parameter is varied between three predetermined levels (learning rate [0.01, 0.001, 0.0001], ratio [0.7, 0.8, 0.9]) to evaluate model performance. When prioritizing overdetection rate, parameter c is recommended; when prioritizing overdetection rate, parameter b is recommended; when prioritizing location accuracy, parameter b is recommended; and for comprehensive evaluation, parameter c, for example, is recommended.

[0044] The retrained model 12 thus generated is automatically subjected to performance evaluation together with the existing trained model 4 by the performance evaluation unit 30. The performance evaluation index 14 used in this case may be the overall precision, recall, and IOU, but preferably the precision, recall, and IOU for each label are used for evaluation.

[0045] Finally, the judgment unit 31 judges whether to replace the learning model using the performance evaluation index 14 provided by the performance evaluation unit 30 to evaluate the performance of the re-learning model.

[0046] (How the retraining model is generated) Next, the operation of the method for generating a re-learning model in this embodiment will be described with reference to the flowchart of FIG.

[0047] Learning step S100: The learning unit 21 performs machine learning on an initial learning dataset 3 including initial image data 1, correct labels, and text data 2 of position coordinates using an object detection algorithm, and generates a learned model 4.

[0048] Prediction step S101: The prediction unit 22 uses the trained model 4 generated by the training unit 21 to infer new image data 5, and obtains an output result for a candidate object in the image as a prediction result 6.

[0049] Prediction result processing step S102: The prediction result processing unit 23 adopts, from among the obtained prediction results 6, prediction results with a confidence level equal to or greater than the first threshold as predictions in which an object has been detected. On the other hand, from among the obtained prediction results 6, the prediction result processing unit 23 rejects prediction results with a confidence level less than the first threshold as predictions in which an object has not been detected. The output of this prediction result processing unit 23 becomes the result of object detection by machine learning.

[0050] Retraining data candidate generation step S103: The retraining data candidate generation unit 24 combines the data whose confidence level is less than the second threshold with the corresponding new image data 5 to form a retraining candidate data set 7.

[0051] Re-learning data selection step S104: The re-learning data selection unit 25 separates the group of objects 8 that should originally be detected from the group of objects 9 that should not be detected, and selects a re-learning data set 10 consisting of the group of objects 8 that should originally be detected. The classification of data for re-learning is determined based on predetermined feature quantities related to the candidate objects. Specifically, the classification based on predetermined feature quantities is determined based on whether a third threshold value such as brightness value, RGB value, or HSV value is set and whether it is equal to or greater than the third threshold value. However, it is effective to perform classification determination after removing noise by binarizing the image data or performing grayscale processing.

[0052] Dataset creation step S105: The dataset creation unit 26 creates a new training dataset 11 by combining the re-training dataset 10 with an existing training dataset.

[0053] Note that the re-learning data candidate generation step S103, the re-learning data selection step S104, the dataset creation step S105, and the configuration for creating a dataset for re-learning are collectively referred to as the re-learning data generation step S106. The dataset creation step S105 is optional, and if a new training dataset 11 is not created, the re-learning data generation step 106 outputs the re-learning dataset 10 to the re-learning step S107.

[0054] Re-learning step S107: The re-learning unit 28 performs re-learning using the re-learning dataset 10 or the new learning dataset 11 to generate a re-learned model 12. The learning parameters 13 used in this step may be the learning parameters used when generating the trained model 4, but preferably, those optimized by the learning parameter optimization unit 29 are used.

[0055] Learning parameter optimization step S108: The learning parameter optimization unit 29 generates various parameter values ​​13a, 13b, etc. that are conditions during learning, generates a learning model using those parameters, and adopts the parameters that provide the best performance using each performance evaluation index 14 of the learning model. This step is optional.

[0056] Performance evaluation step S109: The performance evaluation unit 30 performs performance evaluation of the re-trained model 12 and the existing trained model 4.

[0057] Determination step S110: The determination unit 31 uses the performance evaluation index 14 of the re-trained model 12 and the existing trained model 4 provided by the performance evaluation unit 30 to determine whether to replace the trained model.

[0058] Every time the learning model is re-learned in step S107, the processes of steps S108 to S110 are repeated.

[0059] According to a method for generating a re-training model according to an embodiment of the present disclosure, a re-training data generation step is included in which a candidate re-training dataset is generated by combining new image data corresponding to the prediction results that have a confidence level less than a second threshold and whose feature quantities of the candidate objects satisfy a predetermined standard among the prediction results to be used, and a re-training dataset is created from the candidate re-training dataset. This significantly reduces the burden of the re-training work of a training model, which previously required a great deal of manpower. Furthermore, because the training data can be continuously and automatically re-trained, it is possible to reduce the amount of training data required for generating an initial model, thereby reducing the man-hours required from model development to implementation.

[0060] The above has described embodiments of the relearning model generation device 20 and the relearning model generation method executed by an information processing device. Other embodiments of the present disclosure may also be embodied as a program for causing a computer to execute the method. In this embodiment, the program may be recorded on a computer-readable recording medium. The computer-readable recording medium may include a non-transitory computer-readable medium, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program may be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD (digital versatile disc) or a CD-ROM (compact disc read-only memory), on which the program is recorded. The program may also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program may also be provided as a program product.

[0061] In this embodiment, the object labels and position coordinate data are text data using a markup language such as xml, but they are not limited to text data and may be binary data. Also, in this embodiment, the image data and the text data of the object labels and position coordinate data are treated as a data set, but they do not necessarily have to be treated as a set, and the image data and the text data may be treated separately by linking them with an ID or the like.

[0062] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to be logically inconsistent, and multiple means or steps can be combined or divided into one. [Explanation of symbols]

[0063] 1. Initial image data 2. Text data of correct labels and location coordinates 3. Initial training dataset 4. Pre-trained model 5 New image data 6 Prediction results 7. Candidate dataset for retraining 8 Objects to be detected 9 Objects that should not be detected 10 Retraining Dataset 11 New training dataset 12 Retraining Model 13 Learning parameters 14 Performance evaluation index 20 Retraining model generator 21 Learning Department 22 Prediction Department 23 Prediction result processing section 24 Retraining data candidate generation unit 25 Retraining data selection section 26 Dataset Creation Department 27 Re-learning data generation unit 28 Re-learning section 29 Learning parameter optimization section 30 Performance Evaluation Department 31 Judgment Department

Claims

1. A method for generating a re-learning model for an object detection model in machine learning executed by an information processing device, comprising: a learning step of generating a trained model using an initial training dataset including a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; a prediction step of outputting a prediction result including text data of a label, position coordinates, and confidence level of a candidate object from new image data using the generated trained model; a re-learning data generation step of creating a re-learning data set from the prediction result based on the confidence level and feature amounts of the candidate object included in the new image data; a retraining step of generating a retrained model using the retrained dataset; A method for generating a re-learning model, comprising:

2. The re-learning data generation step includes: a re-training data candidate generation step of generating a re-training data candidate data set for a prediction result whose certainty is less than a second threshold value together with the corresponding new image data; and a re-training data selection step of selecting a re-training data set from prediction results in which the candidate objects included in the new image data of the re-training candidate data set satisfy a criterion related to the feature amount.

3. The method for generating a re-training model according to claim 2 , further comprising a prediction result processing step of determining whether to accept or reject the prediction result based on a first threshold value related to the confidence level.

4. The method for generating a re-training model according to claim 2 , wherein the feature amount is luminance, and the criterion for the feature amount is that the luminance is equal to or greater than a third threshold.

5. The method for generating a re-training model according to claim 2 , wherein in the re-training data selection step, text data of the label or position coordinate of the prediction result can be modified.

6. The method for generating a re-training model according to claim 3 , wherein the prediction result processing step allows the first threshold to be set arbitrarily.

7. The method for generating a retrained model according to claim 6 , wherein the prediction result processing step sets the first threshold value according to performance of the trained model.

8. 3. The method for generating a relearning model according to claim 2, wherein the relearning data candidate generation step at least generates candidate image data by superimposing the new image data including the candidate object of the prediction result having the certainty factor less than a second threshold value and a candidate position of the candidate object generated from text data of the position coordinates of the candidate object of the prediction result, and sets the generated candidate image data as image data included in the relearning candidate dataset.

9. 4. The method for generating a relearning model according to claim 3, wherein the relearning data candidate generation step generates the relearning candidate data set using a prediction result whose confidence is equal to or greater than the first threshold and less than a second threshold.

10. a dataset creation unit that creates a new training dataset including the re-training dataset and the initial training dataset; The method for generating a retrained model according to claim 1 , wherein the retraining step generates a retrained model using the new training data set.

11. 2. The method for generating a retrained model according to claim 1, further comprising a learning parameter optimization step, wherein the learning parameter optimization step generates parameters that optimize the learning conditions when generating the retrained model by the retraining step.

12. The method for generating a retraining model according to claim 11, wherein the learning parameter optimization step randomly generates various parameter values ​​that serve as conditions during learning, and adopts the parameters that result in the best performance.

13. The method for generating a retraining model according to claim 11 , wherein the learning parameter optimization step generates parameter values ​​within a predetermined range, and adopts the parameters that provide the best performance.

14. 14. The method for generating a re-training model according to claim 12 or 13, wherein the parameters include at least one of a learning rate in machine learning using a neural network, a hold-out ratio when dividing all data into training data and validation data in a hold-out method, momentum, and weight decay.

15. 2. The method for generating a retrained model according to claim 1, further comprising a performance evaluation step, wherein the performance evaluation step uses a performance evaluation index of the retrained model to evaluate the performance of the retrained model against an existing trained model.

16. The method for generating a re-training model described in claim 15, wherein the performance evaluation step uses at least one of the precision, recall, and IOU of the learning model as a performance evaluation index, which is determined based on the performance required of the learning model, including at least one of the overdetection rate, overlooked rate, and location accuracy of the learning model, and makes it possible to arbitrarily set thresholds for the performance evaluation indexes of precision, recall, and IOU.

17. 16. The method for generating a re-trained model according to claim 15, further comprising a determination step, in which a determination is made to replace the trained model when the performance evaluation step determines that the performance of the trained model is superior to that of the existing trained model.

18. A device for generating a re-learning model for an object detection model in machine learning, comprising: a learning unit that generates a trained model using an initial training dataset that includes a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; a prediction unit that uses the generated trained model to output a prediction result including text data of a label, position coordinates, and confidence level of a candidate object from new image data; a re-learning data generation unit that creates a re-learning data set from the prediction result based on the confidence level and feature amounts of the candidate object included in the new image data; a re-training unit that generates a re-trained model using the re-training dataset; A re-learning model generation device comprising:

19. A program for generating a retraining model for an object detection model in machine learning, comprising: generating a trained model using an initial training dataset including a plurality of image data to which correct labels of objects and text data of position coordinates are assigned; outputting a prediction result including a label of a candidate object and text data of its position coordinates and confidence level from new image data using the generated trained model; creating a re-learning dataset from the prediction result based on the confidence level and the feature amount of the candidate object included in the new image data; generating a retrained model using the retrained dataset; A program for generating a retraining model that executes the above.

Images