Efficient data image classification method based on multi-tree genetic programming

Abstract

The application discloses a data efficient image classification method based on multi-tree genetic programming, comprising the following steps: obtaining an image to be classified; pre-processing the image to be classified; a genetic programming individual respectively processes red, green and blue channels of the image, and performs multi-scale operation, region extraction, feature extraction and feature connection on the channels; the individual obtained through evolution is used for feature conversion on the image to be classified to obtain new features and normalization; and the new features are input into a classifier to output a classification result of the image to be classified. The color feature extraction method based on the multi-tree genetic programming can learn color features with discrimination and rich information, thereby improving classification accuracy; and the multi-scale feature extraction can enhance the ability of capturing more comprehensive and rich image features for classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a data-efficient image classification method based on multi-tree genetic programming. BACKGROUND

[0002] Image classification refers to the process of classifying / labeling images into predefined categories based on their visual content. It is a fundamental task in computer vision and machine learning, and has various applications in object recognition, autonomous vehicles, and surveillance systems, etc. Data-efficient image classification aims to achieve high classification performance with limited training data. However, when the available training data is insufficient, it becomes more challenging to determine the most relevant and discriminative image features.

[0003] Over the years, many image classification methods have been developed. Convolutional neural networks have achieved remarkable success in image classification. They are composed of multiple convolutional layers and fully connected layers. Convolutional neural networks can learn a hierarchical representation of images, i.e., discriminative image features, and thus are very effective in image classification. However, they often require large datasets to achieve good performance, which can be time-consuming and costly to obtain, annotate, and train. To achieve data-efficient image classification, some techniques such as data augmentation, transfer learning, and semi-supervised learning have been applied to convolutional neural network methods. However, training / running convolutional neural network models requires a large amount of computational resources, thereby limiting their availability in resource-constrained environments. In addition, it is challenging to understand how convolutional neural networks, especially deep architectures, arrive at specific classification decisions, thereby hindering their interpretability.

[0004] In recent years, genetic programming has been successfully applied to image classification. Genetic programming is a subfield of evolutionary computation, which is a machine learning technique inspired by biological evolution. It can automatically evolve computer programs / models to solve complex problems. In genetic programming, programs / individuals / solutions represented by symbolic expressions / trees usually constitute a population. These individuals evolve and optimize through repeated generations, simulating the evolutionary process of selection and genetic operations. In the image classification task, genetic programming has many advantages. First, the genetic programming method for image classification represents the model as a symbolic expression / tree, which provides readability and interpretability of the evolved model, which allows us to understand the underlying logic and facilitate understanding and analysis of the evolved model. Second, overly complex models may have difficulty generalizing from limited training data. The evolved genetic programming model usually has a relatively simple structure (less tree depth and node number) and a small number of parameters, which may produce a model with good generalization ability. Finally, the genetic programming-based image classification method has the advantage of training and executing on CPU without the need for expensive computing resources. Due to the above advantages, genetic programming methods have shown good performance in image classification.

[0005] In the process of implementing the present application, the inventors found that the prior art has the following technical problems:

[0006] Due to the following limitations, the potential of genetic programming in image classification still needs further research. First, most existing genetic programming methods use grayscale images as input to genetic programming trees and learn features from them. However, color is a basic feature of image representation and may play an important role in image classification. Therefore, it is necessary to explore methods that use genetic programming to directly learn features from color images to solve image classification. Existing genetic programming methods usually only focus on learning features on a single scale image, which limits the potential improvement of accuracy. Multi-scale feature extraction is a strategy that can capture valuable information at different levels of detail / scale within an image and has been successfully applied to convolutional neural networks, enhancing their feature learning ability and improving classification accuracy. SUMMARY

[0007] The present application provides a data-efficient image classification method based on multi-tree genetic programming to learn information-rich and discriminative image features from a small number of training instances, achieving high classification accuracy.

[0008] The technical scheme adopted by the present application is:

[0009] The data-efficient image classification method based on multi-tree genetic programming comprises the following steps:

[0010] S1, obtaining an image to be classified;

[0011] S2, pre-processing the image to be classified;

[0012] S3, the genetic programming individual respectively processes the red, green and blue color channels of the image, and performs multi-scale operation, region extraction, feature extraction and feature connection;

[0013] S4, the evolved individual is used to convert the features of the image to be classified to obtain new features and normalize them;

[0014] S5, input the new features into the classifier to output the classification result of the image to be classified.

[0015] Further, the specific steps of S3 are:

[0016] S331: input the image to be classified;

[0017] S332: design the program structure, function set and terminal set of genetic programming respectively, each individual contains three trees, the program structure and function set of the three trees are the same, and the input terminal set is the red, green and blue color channels of the image respectively;

[0018] S333: generate genetic programming tree individuals by the input-output correspondence of each node, and initialize the population;

[0019] S334: evaluate the fitness value of each individual in the population according to the selected individual fitness evaluation function;

[0020] S335: select parent individuals through elite operation and tournament selection, and then generate next generation individuals through crossover and mutation genetic operations;

[0021] S336: evaluate the fitness value of each individual in the new population using the fitness evaluation function in S334;

[0022] S337: determine whether the maximum number of iterations of the population is reached; if yes, stop evolution; otherwise, go to S336 to continue the search process.

[0023] Further, the program structure in S332 is:

[0024] Each individual includes three trees, which respectively process the red, green and blue color channels of the image; the structure of the tree is composed of multiple layers with different functions; these layers perform the subtasks of image classification in order, i.e. multi-scale operation, region detection, feature extraction and feature connection; among them, the multi-scale operation layer and the region detection layer are optional;

[0025] The function set includes multi-scale operation functions, region selection functions, feature extraction functions and feature connection functions; the multi-scale operation function U_S generates a high-resolution image using a bicubic interpolation method, and the D_S function constructs a low-resolution image using the same technique;

[0026] The region selection functions Region_S and Region_R can automatically identify square and rectangular regions in the image; by setting the position and size of the detected image region as the terminal nodes of the genetic programming tree, the genetic programming method can effectively identify appropriate parts of the image that distinguish different categories, thereby extracting local features useful for classification;

[0027] The feature extraction functions can capture various image features such as shape and texture features, including G_Hist and L_Hist that extract histogram features from the entire image and the detected image region, respectively; G_uLBP and uLBP that extract global or local binary pattern features; G_SIFT and L_SIFT that extract global and local scale-invariant feature transform features; G_HOG and L_HOG that extract global and local gradient histogram features; and G_DIF and L_DIF that extract global and local field-independent features;

[0028] The feature connection functions FeaCon2 and FeaCon3 combine two / three feature vectors obtained from the feature extraction or connection layer into one feature vector by concatenation;

[0029] The terminal set includes: in order to extract color features from the image, the method introduces three new terminals, namely blue, green and red, as inputs to the three trees in a single genetic programming individual; blue, green and red are two-dimensional arrays, in which the pixel values are normalized to the interval [0, 1]; X and Y represent the coordinates of the top-left corner of the detected image region; Size represents the size of the square region detected by the region; Height and Width represent the height and width of the rectangular region detected by the region.

[0030] Further, step S333 is specifically:

[0031] The size of the genetic programming population is set to 100, the number of iterations is set to 50, the generation method of the individual is "Ramped half-and-half", the depth range of the tree is [4, 8] for the first generation, and the maximum depth of the tree is 8 thereafter.

[0032] Further, step S334 is specifically:

[0033] An individual is fed a training set into a genetic programming individual and transformed into features. The transformed features are then normalized using a min-max normalization method. The normalized features and class labels are fed into a linear support vector machine, and hierarchical K-fold cross-validation is used to train and evaluate each genetic programming individual. The training set is divided into K folds. In each iteration, K-1 folds, called the evaluation training set, are used to train the classifier, and the remaining folds, called the evaluation test set, are used to test the constructed classifier. This evaluation process is repeated K times, with each fold used only once as the evaluation test set. The fitness of each genetic programming individual is determined by averaging the accuracy across the K evaluation test sets.

[0034] Furthermore, the specific steps of S335 include:

[0035] S3351: Elite Operation;

[0036] The elite operator directly uses the individual with the highest fitness value as the offspring;

[0037] S3352: Selection operation;

[0038] Tournament selection is used as a parent selection method to select promising parent individuals for generating new offspring individuals; in tournament selection, an individual is selected as a parent based on its overall fitness value.

[0039] The beneficial effects of this invention are:

[0040] 1. Use multiple trees to extract features from different color channels to improve the classification performance of efficient image classification.

[0041] 2. It has the advantages of simple implementation, low cost, high classification accuracy and few adjustable parameters;

[0042] 3. The genetic programming method for image classification represents the model as a symbolic expression / tree, which improves the readability and interpretability of the model;

[0043] 4. The parent selection strategy adopts tournament selection, which selects promising parent individuals to generate new offspring individuals; in tournament selection, an individual is selected as the parent based on its overall fitness value. Attached Figure Description

[0044] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0045] The invention will now be further described with reference to the accompanying drawings.

[0046] like Figure 1 As shown, the efficient image classification method based on multi-tree genetic programming includes the following steps:

[0047] S1: obtaining an image to be classified.

[0048] S2: preprocessing the image to be classified, including modifying the picture size, etc.

[0049] S3: genetic programming individuals respectively process the red, green and blue channels of the image, and perform multi-scale operation, region extraction, feature extraction and feature connection, the specific steps are as follows:

[0050] S331: inputting an image to be classified.

[0051] S332: designing the program structure, function set and terminal set of genetic programming respectively, each individual contains three trees, the program structure and function set of the three trees are the same, and the input terminal set is the red, green and blue channels of the image.

[0052] In the program structure, each individual includes three trees, which respectively process the red, green and blue channels of the image. The structure of the tree is composed of multiple layers with different functions, which perform subtasks of image classification in order, i.e. multi-scale operation, region detection, feature extraction and feature connection. Among them, the multi-scale operation layer and the region detection layer are optional, which makes the present application adapt to learning the features of the original image and / or the modified image (such as the up-sampled or down-sampled version). In addition, the present application also has the flexibility to learn global and / or local features from the image.

[0053] The function set includes multi-scale operation function, region selection function, feature extraction function and feature connection function. The multi-scale operation function U_S generates a high-resolution image (i.e. the original image size x 2) using bicubic interpolation method, and the D_S function constructs a low-resolution image (i.e. the original image size x 1 / 2) using the same technique. The region selection functions Region_S and Region_R can automatically identify square and rectangular regions in the image. By setting the position and size of the detected image region as the terminal node of the genetic programming tree, the genetic programming method can effectively identify the appropriate part of the image that distinguishes different categories, thereby extracting local features useful for classification.

[0054] The feature extraction function can capture various image features such as shape and texture features, including G_Hist and L_Hist to extract histogram features from the entire image and the detected image region respectively; G_uLBP and uLBP to extract global or local binary pattern features; G_SIFT and L_SIFT to extract full and local scale invariant feature transform features; G_HOG and L_HOG to extract full and local gradient histogram features; G_DIF and L_DIF to extract full and local field independent features.

[0055] The feature connection functions FeaCon2 and FeaCon3 merge two / three feature vectors obtained from the feature extraction or connection layer into one feature vector by concatenation.

[0056] The terminal set, in order to extract color features from images, introduces three new terminals, namely blue, green and red, as inputs to the three trees in a single genetic programming individual. Blue, green, red are two-dimensional arrays, in which the pixel values are normalized to be in the interval [0, 1]. X and Y represent the coordinates of the top-left point of the detected image region. Size represents the size of the detected region for a square region, and Height and Width represent the height and width of the detected region for a rectangle region.

[0057] S333: Generate genetic programming tree individuals by the input-output correspondence of each node, and initialize the population. The size of the genetic programming population is set to 100, the number of iterations is set to 50, the generation method of the individual is "Ramped half-and-half", the depth range of the tree is [4, 8] for the first generation, and the maximum depth of the tree is 8 thereafter.

[0058] S334: According to the selected individual fitness evaluation function, evaluate the fitness value of each individual in the population; input a set of image training set into the genetic programming individual and convert it into features; then use the min-max normalization method to normalize the converted features; the normalized features and class labels are input into the linear support vector machine, and stratified K-fold cross-validation is used to train and evaluate each genetic programming individual. The training set is divided into K folds, and in each iteration, K-1 folds called evaluation training set are used to train the classifier, and the remaining folds called evaluation test set are used to test the constructed classifier. This evaluation process is repeated K times, and each fold is used only once as the evaluation test set. The fitness of each genetic programming individual is determined by averaging the accuracy rate in the K evaluation test sets. Due to the limitation of small training sets, the value of K is set to 3.

[0059] S335: Select parent individuals through elitist operation and tournament selection, and then generate the next generation of individuals through crossover and mutation genetic operations;

[0060] S3351: Elitist operation;

[0061] The elitist operator directly takes the individual with the highest fitness value as the offspring.

[0062] S3352: Selection operation;

[0063] The tournament selection is used as the parent selection method to select promising parent individuals for generating new offspring individuals; in the tournament selection, one individual is selected as a parent according to its overall fitness value. The fitness value is, for example, the classification accuracy on all training instances.

[0064] S3352: crossover operation;

[0065] Two individuals selected by the selection operation, using subtree crossover, exchanging the nodes of the same input and output of two trees, generating two offspring from two parents;

[0066] S3353: mutation operation;

[0067] Select a node of a tree for subtree mutation, randomly generate a subtree replacement branch rooted at the node according to the input and output of the node, and generate a new tree.

[0068] S336: evaluate the fitness value of each individual in the new population using the fitness evaluation function in S334;

[0069] S337: determine whether the maximum number of iterations of the population is reached; if yes, stop evolution; otherwise, go to S336 to continue the search process.

[0070] S4: the individuals evolved are used to convert the features of the image to be classified to obtain new features and normalize them;

[0071] S5: input the new features into the classifier to output the classification result of the image to be classified.

[0072] In order to further illustrate the superiority of the present application in dealing with face recognition problems, Table 1 shows the final classification accuracy on the test set obtained by the present application and the convolutional neural network method MobileNetV3 and two genetic programming-based methods in solving Amazon, Webcam, Dslr and Aberdeen four classification data sets.

[0073] In the example, four classic classification problems are given as examples, each classification problem is independently executed 30 times, the size of the individual in the population is set to 100, the maximum number of iterations is set to 50, and the performance index value obtained each time is recorded. Table 1 shows the average accuracy and standard deviation, each block in the table shows all the results on one data set, and the best result is highlighted in bold.

[0074] Table 1 Comparison of classification accuracy of Amazon, Webcam, DSLR and Aberdeen classification problems

[0075]

[0076] According to Table 1, the application is obviously superior to the benchmark method in most comparisons. Compared with MobileNetV3, the average accuracy of the application on the four datasets is significantly improved by 22.06%, 12.18%, 15.11% and 27.12% respectively. According to the classification results, the convolutional neural network comparison method is obviously inferior to the application in effect in most cases. One possible explanation is that the convolutional neural network usually involves a large number of parameters, and a large amount of training data is needed to effectively train the model. In contrast, the application has a relatively simple tree-based structure, only uses some functions and terminals representing solutions, and can learn useful features from limited training instances. Compared with the two genetic programming methods, the application obtains significantly higher classification accuracy. These results confirm the effectiveness of the application in the data-efficient image classification task.

[0077] In summary, the data-efficient image classification technology based on multi-tree genetic programming proposed in the application can obtain good performance in actual image classification.

[0078] Evolutionary algorithms have been widely proven to have strong global search ability and can effectively search for global optimal or near-optimal solutions in complex spaces. As a class of evolutionary algorithms, genetic programming has been widely used to solve various practical problems due to its high efficiency and strong interpretability. Genetic programming, as a subfield of evolutionary computation, can automatically evolve computer programs / models to solve complex problems. Genetic programming combines multiple levels of feature transformations by using various powerful image-related operators as functions in tree structures. Even in the case of limited training images, informative features can be learned.

[0079] Therefore, the application proposes a new data-efficient image classification method based on multi-tree genetic programming, which improves the classification accuracy by learning efficient features in complex images using multi-tree genetic programming.

[0080] The above describes the specific embodiments of the application in conjunction with the drawings, but is not a limitation on the protection scope of the application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art without creative labor on the basis of the technical solutions of the application are still within the protection scope of the application.

Claims

1. A data-efficient image classification method based on multi-tree genetic programming, characterized in that, The method comprises the following steps: S1, obtaining an image to be classified; S2, preprocessing the image to be classified; S3, a genetic programming individual respectively processes red, green and blue color channels of the image, and performs multi-scale operation, region extraction, feature extraction and feature connection on the image; the specific steps of S3 are: S331: inputting an image to be classified; S332: designing a program structure, a function set and a terminal set of the genetic programming respectively, each individual comprising three trees, the program structures and the function sets of the three trees being the same, and the input terminal sets being red, green and blue color channels of the image respectively; S333: generating a genetic programming tree individual through an input-output correspondence of each node, and initializing a population; S334: evaluating an adaptability value of each individual in the population according to a selected individual adaptability evaluation function; specifically, the individual inputs a training set into the genetic programming individual and converts the training set into features; then, the converted features are normalized by using a min-max normalization method; the normalized features and class labels are input into a linear support vector machine, and stratified K-fold cross-validation is used to train and evaluate each genetic programming individual; the training set is divided into K folds, in each iteration, K-1 folds, referred to as evaluation training sets, are used to train the classifier, and the remaining fold, referred to as an evaluation test set, is used to test the constructed classifier; the above evaluation process is repeated K times, and each fold is used only once as the evaluation test set; the adaptability of each genetic programming individual is determined by averaging the accuracy rates in the K evaluation test sets; S335: selecting parent individuals through elitist operation and tournament selection, and generating next-generation individuals through genetic operations such as crossover and mutation; S336: evaluating an adaptability value of each individual in the new population by using the adaptability evaluation function in S334; S337: determining whether the maximum iteration number of the population is reached; if yes, stopping evolution; otherwise, turning to S336 to continue the search process; S4: converting features of the image to be classified by using the evolved individual to obtain new features, and normalizing the new features; S5: inputting the new features into a classifier to output a classification result of the image to be classified.

2. The method of claim 1, wherein, The program structure in S332 is: each individual comprises three trees, which respectively process red, green and blue color channels of the image; the structure of the tree is composed of multiple layers with different functions; these layers sequentially execute subtasks of image classification, i.e., multi-scale operation, region detection, feature extraction and feature connection; among them, the multi-scale operation layer and the region detection layer are optional; the function set comprises a multi-scale operation function, a region selection function, a feature extraction function and a feature connection function; the multi-scale operation function U_S generates a high-resolution image by using a bicubic interpolation method, and the D_S function constructs a low-resolution image by using the same technology; The region selection functions Region_S and Region_R can automatically identify square and rectangular regions in the image; by setting the position and size of the detected image regions as the terminal nodes of the genetic programming tree, the genetic programming method can effectively identify the appropriate parts of the image that distinguish different categories, thereby extracting local features useful for classification; The feature extraction functions can capture various image features such as shape and texture features, including G_Hist and L_Hist to extract histogram features from the entire image and the detected image region, respectively; G_uLBP and uLBP to extract global or local binary pattern features; G_SIFT and L_SIFT to extract global and local scale invariant feature transform features; G_HOG and L_HOG to extract global and local gradient histogram features; G_DIF and L_DIF to extract global and local field-independent features; The feature concatenation functions FeaCon2 and FeaCon3 merge two / three feature vectors obtained from the feature extraction or concatenation layer into one feature vector by concatenation; In order to extract color features from the image, the method introduces three new terminals, namely blue, green and red, as inputs to the three trees in a single genetic programming individual; blue, green and red are two-dimensional arrays, in which the pixel values are normalized to the interval [0, 1]; X and Y represent the coordinates of the top-left corner of the detected image region; Size represents the size of the square region detected by the region; Height and Width represent the height and width of the rectangular region detected by the region.

3. The method of claim 1, wherein, Step S333 is specifically: The size of the genetic programming population is set to 100, the number of iterations is set to 50, the generation method of the individual is "Ramped half-and-half", the depth range of the tree is [4, 8] in the first generation, and the maximum depth of the tree is 8 thereafter.

4. The method of claim 1, wherein, The specific steps of S335 include: S3351: elite operation; The elite operator directly selects the individual with the highest fitness value as the offspring; S3352: selection operation; The tournament selection is used as the parent selection method to select promising parent individuals for generating new offspring individuals; in the tournament selection, one individual is selected as the parent according to its overall fitness value.

Images