An image labeling method based on autoencoder

Abstract

The present invention discloses an image labeling method based on an autoencoder, which belongs to the field of image labeling technology and includes a sample image, wherein the sample image contains a large number of images with different types of features. The sample image is input into an autoencoder, and the autoencoder classifies each of the sample images into the same first-level classification according to its features. The sample images in the same first-level classification are further classified into the same second-level classification according to their features, wherein the same sample image may be classified into different first-level classifications or second-level classifications at the same time. The autoencoder obtains a corresponding label through each classification, and the image to be labeled is input into the autoencoder. The beneficial effect of the present invention is that during the recognition process of an image with a custom label, the custom label will not be replaced by a newly determined label, and the image classification method is adaptively adjusted according to the image type. The method has strong flexibility and can improve the accuracy of the label prediction results.

Description

Technical Field

[0001] The present invention relates to the field of image labeling, and in particular to an image labeling method based on an autoencoder. Background Art

[0002] Autoencoders are a type of artificial neural network used in supervised learning and unsupervised learning. Their function is to represent the output learning by taking the input information as the learning target. The autoencoder consists of two parts, the encoder and the decoder. The autoencoder can be divided into contraction autoencoders, regularized autoencoders and variational autoencoders. The strong two are discriminant models, and the latter is a generative model. The autoencoder can make a neural network with a feedforward structure or a recursive structure. The autoencoder has the function of a representation learning algorithm in a general sense and is used for dimensionality reduction and outlier detection. The autoencoder constructed with convolutional layers can be applied to computer vision problems, including image denoising, neural style transfer, etc.

[0003] With the rapid development of the Internet, image recognition technology has been widely used in many scenarios around us. Image recognition is used to identify images, and labels are added to images based on the recognition results to express the characteristic content of the image, so that users can use images to search for things related to the image content. For example, an image containing fish can be labeled "fish", "grass carp", "one of the four major carps" and so on after scanning. However, when the existing technology recognizes an image with a custom label, it will be replaced by the newly determined label. In real usage scenarios, many images do not need too many labels. Too many labels will cause trouble to users. In addition, too many labels require stronger computing power, occupy more computing resources, and take longer to calculate, which will reduce efficiency.

[0004] The above prior art has the following problems:

[0005] (1) During the recognition process of images with custom labels, the custom labels will be replaced by the newly determined labels;

[0006] (2) When existing algorithms add labels to many images that do not require too many labels, they will add too many labels, which will take up too much computing resources, take up too much computing time, and cause inconvenience to users. Summary of the Invention

[0007] The purpose of the present invention is to provide an image labeling method based on an autoencoder to solve the problems raised in the above background technology.

[0008] The technical solution of the present invention is: including sample images, wherein the sample images contain a large number of images with different types of features, the sample images are input into an automatic encoder, the automatic encoder classifies each of the sample images into the same first-level classification according to its features, and the sample images in the same first-level classification are further classified into the same second-level classification according to their features, wherein the same sample image may be classified into different first-level classifications or second-level classifications at the same time, the automatic encoder obtains corresponding labels through each classification, the image to be labeled is input into the automatic encoder, the image is processed by the automatic encoder for dimensionality reduction, and the feature map of the image is obtained, the feature map has a corresponding feature vector for each classification, and each feature vector under the classification contains multiple points, and each point has a probability value with the corresponding classification, according to the probability value. The probability value determines the top N labels with the highest probability values as the labels to be added for the image, where N can be a preset value not less than 1. It is determined whether the image has a custom label. If the image has a custom label and the labels to be added exist in the labels to be added, the custom label is retained, and the other labels except the custom label in the current labels to be added for the image are updated as the image labels to be added for this time. The number of labels to be added corresponding to the image to be labeled is A, and the value of A is a preset value not less than 1. The number of labels to be added + the number of custom labels = A. When the number of labels to be added + the number of custom labels is greater than A, the labels to be added are deleted starting from the label with the smaller probability until the number of labels to be added + the number of custom labels = A.

[0009] Furthermore, before the autoencoder classifies the sample images, the method also includes establishing corresponding sample images for each classification; initializing the image label classification model; and performing classification training on the autoencoder based on the labels corresponding to the sample images in each classification.

[0010] Furthermore, the image processing method further includes analyzing the remark information of the image when receiving the image, and adding the remark information as a custom tag of the image.

[0011] Furthermore, the custom tag sets a preset first identifier, and detecting whether there is a tag to be added that overlaps with the custom tag in the tags to be added includes detecting whether there is an image tag with the first identifier in the tags to be added. If there is a tag to be added with the first identifier among all the tags to be added, it is determined that there is a tag to be added with the first identifier in the tags to be added. If there is no tag to be added with the first identifier among all the tags to be added, it is determined that there is no tag to be added with the first identifier in the tags to be added.

[0012] Furthermore, after detecting whether there is a custom tag in the tags to be added, if there is a custom tag in the tags to be added, the custom tags in all the tags to be added are taken out, and the remaining tags to be added are positioned as image tags. If there is no custom tag in the image to be added, the tag to be added with the lowest probability value among all the tags to be added is removed, and the remaining tags to be added are set as tags of the image.

[0013] Furthermore, the step of inputting the image to be labeled into the autoencoder to obtain a feature map includes inputting the image to be labeled into the autoencoder, determining the first feature map of the icon to be labeled, performing dimensionality reduction processing on the first feature map to obtain a second feature map; performing average pooling on the second feature map to obtain a feature vector corresponding to the second feature map.

[0014] The present invention provides an image labeling method based on an autoencoder through improvements, which has the following improvements and advantages compared with the prior art:

[0015] First, the present invention sets a preset first identifier for the custom tag, and detects whether there is a tag to be added that overlaps with the custom tag in the tags to be added, including: detecting whether there is an image tag with the first identifier in the tags to be added; if there is a tag to be added with the first identifier in all the tags to be added, then it is determined that there is a tag to be added with the first identifier in the tags to be added; if there is no tag to be added with the first identifier in all the tags to be added, then it is determined that there is no tag to be added with the first identifier in the tags to be added; after detecting whether there is a custom tag in the tags to be added, if there is a custom tag in the tags to be added, the custom tags in all the tags to be added are removed, and the remaining tags to be added are positioned to cover the image tag; if there is no custom tag in the image to be added, the tag to be added with the lowest probability value among all the tags to be added is removed, and the remaining tags to be added are set as the tags of the changed image, so that the custom tag will not be replaced by the newly determined tag during the recognition process of the image with the custom tag;

[0016] Secondly: the present invention uses a convolutional neural network to minimize the cross entropy loss function for the trained sample set using a back propagation algorithm to adjust the weights of the convolutional neural network for training, and reloads the weights of the trained convolutional neural network to extract the sample network feature set of the training sample set and the test network feature set of the image to be labeled; according to the sample network feature set, the test network feature set and the label set, the probability that the image to be labeled belongs to each type of label in the label set is calculated to generate a label probability set; finally, according to the label probability set, the image to be labeled is labeled by training one or more autoencoders in advance to train a label recognition model containing multiple semantic hierarchical classifiers, inputting the image into the convolutional neural network, determining the feature map of the image through the convolutional neural network, and inputting the feature map into each class in the multi-label model. In the semantic level classifier, each semantic level classifier predicts and outputs a label that matches the image. Since each semantic level classifier represents a different semantic level, the output is also a label representing a different level, so as to accurately identify the category to which the image belongs. The image is input into the autoencoder of the convolutional architecture to determine the feature map of the image, and the corresponding labels of the feature map are classified and analyzed. A first-level classifications with the highest probability values are selected, and then A second-level classifications are selected from these first-level classifications respectively. Then, A labels with the largest probability are selected from these first-level classifications and second-level classifications. Therefore, when making predictions, the corresponding classification tasks are adaptively mobilized according to the image type to perform label prediction, thereby achieving the purpose of adaptively adjusting the image classification method according to the image type. It is highly flexible and can improve the accuracy of label prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further explained below in conjunction with the accompanying drawings and examples:

[0018] Figure 1 This is a flowchart of the steps of an image labeling method based on an autoencoder of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of each step of an image labeling method based on an autoencoder in the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the Figures 1 to 2 The present invention is described in detail, and the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] The present invention provides an image labeling method based on an autoencoder by improving Figure 1-Figure 2As shown, including sample images, the sample images contain images of a large number of different types of features, the sample images are input into the autoencoder, the autoencoder classifies each sample image into the same first-level classification according to its features, and the sample images in the same first-level classification are further classified into the same classification according to their features, wherein the same sample image may be classified into different classifications at the same time, the autoencoder obtains the corresponding label through each classification, the autoencoder inputs the image to be labeled, the image is processed by the autoencoder for dimensionality reduction, and the feature map of the image is obtained, the feature map and each classification have corresponding feature vectors, each feature feature vector under the classification contains multiple points, each point has a probability value with the corresponding classification, and the top N labels with the highest probability values are determined as the labels to be added to the image according to the probability value, wherein N can be a preset value not less than 1, and the image is judged. Whether there is a custom label. If there is a custom label in the image and there is a custom label in the label to be added, then the custom label is retained, and the other labels except the custom label in the current label to be added of the image are updated to the image label to be added of this image. The final number of labels to be added corresponding to the image to be labeled is A, and the value of A is a preset value not less than 1. The number of labels to be added + the number of custom labels = A. When the number of labels to be added + the number of custom labels is greater than A, the labels with smaller probability pins are deleted from the labels to be added until the number of labels to be added + the number of custom labels = A. In this way, during the recognition process of images with custom labels, the custom labels will not be replaced by the newly determined labels. The purpose of adaptively adjusting the image classification method according to the image type is highly flexible and can improve the accuracy of the label prediction results.

[0022] Before the autoencoder classifies the sample images, the method also includes establishing corresponding sample images for each classification; initializing the image label classification model; and performing classification training on the autoencoder based on the labels corresponding to the sample images in each classification, so as to facilitate the autoencoder to classify the images to be labeled in the later stage.

[0023] The image processing method further includes analyzing the remark information of the image when receiving the image, adding the remark information as a custom tag of the image, and confirming whether the image to be tagged has the custom tag.

[0024] The custom tag sets a preset first identifier, and detecting whether there is a tag to be added that overlaps with the custom tag in the tags to be added includes detecting whether there is an image tag with the first identifier set in the tags to be added. If there is a tag to be added with the first identifier set in all the tags to be added, it is determined that there is a tag to be added with the first identifier in the tags to be added. If there is no tag to be added with the first identifier set in all the tags to be added, it is determined that there is no tag to be added with the first identifier in the tags to be added, and it is determined whether the tags to be added contain the custom tag.

[0025] After detecting whether there is a custom label in the labels to be added, if there is a custom label in the labels to be added, the custom labels in all the labels to be added are removed, and the remaining labels to be added are positioned to cover the image labels. If there is no custom label in the image to be added, the label to be added with the lowest probability value among all the labels to be added is removed, and the remaining labels to be added are set as the labels of the image, which facilitates the removal of custom labels in the labels to be added.

[0026] The step of inputting the image to be labeled into the autoencoder to obtain a feature map includes inputting the image to be labeled into the autoencoder, determining a first feature map of the icon to be labeled, performing dimensionality reduction processing on the first feature map to obtain a second feature map; performing average pooling on the second feature map to obtain a feature vector corresponding to the second feature map, and obtaining the feature map facilitates the subsequent acquisition of the label to be added for the image to be labeled.

[0027] Working principle: By training multiple autoencoders, a label recognition model containing multiple semantic level classifiers is trained, and the image is input into the convolutional neural network. The feature map of the image is determined by the convolutional neural network, and the feature map is input into each semantic level classifier in the multi-label model. Each semantic level classifier predicts and outputs a label that matches the image. Since each semantic level classifier represents a different semantic level, the output also represents a label of a different level, thereby accurately identifying the category to which the image belongs. Step 1: Input the sample into the autoencoder. Determining the feature map includes: training one or more autoencoders, inputting a large number of sample images into the autoencoder of the convolutional architecture for dimensionality reduction processing, and obtaining the feature of the sample image. Figure, so that the autoencoder uses the sample's feature map for classification training. During the training process, a large number of sample feature maps are divided into multiple first-level categories, and then the sample feature maps in the same first-level category are divided into pairs of second-level categories. For some sample feature maps with only one label, they are independently placed in the corresponding first-level category. Step 2: The image to be labeled is input into the autoencoder, and the image is recognized based on a preset image recognition algorithm to obtain a feature vector of the image. The obtained feature vector of the image is input into a pre-trained label image model to obtain the confidence of the label image model, and the top N text labels with the highest confidence are determined as the image label to be added to the image. Step 3: The above-mentioned N can be a preset value of not less than 1. Alternatively, the image can be segmented first to obtain at least one segmented image of the image. Based on a preset image recognition algorithm, each segmented image of the image is recognized separately to obtain a feature vector of each segmented image. The feature vector of each segmented image is input into a pre-trained label image model to obtain a probability value of the label image model, and the text label with the highest probability value is determined as the image label of the corresponding segmented image. The image label of the image is determined by combining the image labels of each segmented image of the image.Specifically, the above-mentioned process of identifying the image based on the preset image recognition algorithm to determine the image label to be added to the above-mentioned image can be implemented with reference to the existing technology and will not be repeated here. Step 4: First, determine whether the image has a custom label. If there is no custom label, locate the label of the image with the label to be added. If it has a custom label, first analyze whether there is a label in the label to be added that overlaps with the custom label. If not, start to determine whether the total number of the label to be added and the custom label is less than or equal to A. If so, remove the custom label from the label to be added. Then, determine whether the total number of the label to be added and the custom label is less than A. If it is greater than A, delete the label to be added, starting with the one with the smaller probability value, until the number of labels to be added and the number of custom labels added is less than or equal to A. Then, adaptively mobilize the corresponding classification task according to the image type to perform label prediction, thereby achieving the purpose of adaptively adjusting the image classification method according to the image type. It is highly flexible and can improve the accuracy of the label prediction results.

[0028] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image labeling method based on an autoencoder, characterized by: Including sample images, the sample images contain a large number of images with different types of features, the sample images are input into the automatic encoder, the automatic encoder classifies each of the sample images into the same first-level classification according to its features, and the sample images in the same first-level classification are further classified into the same second-level classification according to their features, wherein the same sample image is classified into different first-level classifications or second-level classifications at the same time, the automatic encoder obtains the corresponding label through each classification, the image to be labeled is input into the automatic encoder, the image is processed by the automatic encoder for dimensionality reduction, and the feature map of the image is obtained, the feature map has a corresponding feature vector for each classification, each feature vector under the classification contains multiple points, each point has a probability value with the corresponding classification, and the probability is calculated according to the probability value. The top N tags with the highest values are determined as the tags to be added for the image, where N is a preset value not less than 1. It is determined whether the image has a custom tag. If the image has a custom tag and the tags to be added include a custom tag, the custom tag is retained, and the tags other than the custom tag in the current tags to be added for the image are updated as the tags to be added for the image. The number of tags to be added corresponding to the tag to be added is A, and the value of A is a preset value not less than 1. The number of tags to be added + the number of custom tags = A. When the number of tags to be added + the number of custom tags is greater than A, the tags to be added are deleted starting from the tags with smaller probabilities until the number of tags to be added + the number of custom tags = A. The custom tag is set with a preset first identifier, and detecting whether there is a tag to be added that overlaps with the custom tag in the tags to be added includes detecting whether there is an image tag with the first identifier set in the tags to be added, if there is a tag to be added with the first identifier set in all the tags to be added, then determining that there is a tag to be added with the first identifier in the tags to be added, and if there is no tag to be added with the first identifier set in all the tags to be added, then determining that there is no tag to be added with the first identifier in the tags to be added; After detecting whether there is a custom tag in the tags to be added, if there is a custom tag in the tags to be added, the custom tags in all the tags to be added are removed, and the remaining tags to be added are positioned as image tags. If there is no custom tag in the image to be added, the tag with the lowest probability value among all the tags to be added is removed, and the remaining tags to be added are set as the label of the image.

2. The image labeling method based on an autoencoder according to claim 1, characterized in that: Before the autoencoder classifies the sample images, the method further includes: establishing corresponding sample images for each classification; initializing the image label classification model; and performing classification training on the autoencoder based on the labels corresponding to the sample images in each classification.

3. The image labeling method based on an autoencoder according to claim 1, characterized in that: The method further includes analyzing the remark information of the image when receiving the image, and adding the remark information as a custom tag of the image.

4. The image labeling method based on an autoencoder according to claim 1, characterized in that: The step of inputting the image to be labeled into the autoencoder to obtain a feature map includes inputting the image to be labeled into the autoencoder, determining a first feature map of the image to be labeled, performing dimensionality reduction processing on the first feature map to obtain a second feature map; and performing average pooling on the second feature map to obtain a feature vector corresponding to the second feature map.

Images