Method, system and computer-readable storage device for constructing image data set

Abstract

The present application discloses a method, system, and computer-readable storage medium for constructing an image data set, wherein the method includes: dividing the pre-obtained first target hash value set to obtain corresponding hash value subsets; wherein, hash The hash value subset has M hash values, and M is an integer greater than or equal to 1; N hash values ​​are extracted from any hash value subset respectively to generate the first target hash value subset; where N is A positive integer less than or equal to M; calculate the union of all first target hash value subsets to obtain a second target hash value set, use the second target hash value set to obtain the corresponding image in the original image, and Constructing the target image data set; the process of obtaining the first target hash value set includes: obtaining the original image data set, obtaining the corresponding original image; calculating the hash value of the original image, and eliminating duplicate hashes according to the calculated hash value value to obtain the first target hash value set. This application implements the construction of differentiated image datasets.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to an image data set construction method, system and computer-readable storage device. Background technique [0002] Data, algorithms, and computing power are the three pillars of machine learning. Data has a huge impact on the performance of machine learning models, and sufficient data is the basis for training machine learning models with good performance. The adequacy of data is not only reflected in the amount of data, but also in the diversity of data. Differentiated data is a more comprehensive description of the problem, and a large number of identical or similar data is just a repeated description of a certain aspect of the problem. For example, UAV inspections of power transmission lines usually collect a large number of images, and many factors lead to many identical and similar images: (1) For the inspection of multiple circuits on the same tower, the exact sa...

AI Technical Summary

Problems solved by technology

Therefore, even if a large amount of data is used to train the machine learning model, because it contains a large amount of the same and similar data, it not only wastes the cost of data labeling, but more importantly, it is difficult to train a machine learning model with good performance.

Images