An image classification method based on orientation features

Abstract

The application discloses an image classification method based on orientation features. The method comprises the following steps: grayscale processing and remodeling of an image to a pixel scale of 225*225; using four 3*3 orientation operators to respectively calculate the cosine similarity of 3*3 slices in the image after the original step S1 remodeling, to obtain four 75*75 size similarity matrices; single threshold masking of each similarity matrix; feature fusion of the orientation feature information obtained by calculating the cosine similarity of the four channels; and feeding the fused feature map into a backbone network for classification. The input based on which the image classification method provided by the application is the orientation feature information of an image, and the image classification can be performed on a grayscale image. Compared with the traditional method of directly feeding an original image into a neural network for classification, the application can significantly reduce the data volume, network size and computing cost of the whole system from the source, and the performance on the classification task is basically equivalent to that of the traditional method.

Description

Technical Field

[0001] This invention relates to an image classification method based on orientation features, belonging to the field of image classification. Background Technology

[0002] Image classification refers to image processing methods that distinguish different categories of objects based on the different features they reflect in image information. It uses computers to quantitatively analyze images, classifying each pixel or region within an image into one of several categories, thus replacing human visual interpretation. Image classification is the core of computer vision, the foundation for other higher-level visual tasks such as object tracking and behavior analysis, and has wide applications in many fields.

[0003] Current image classification tasks directly input color or grayscale images captured by image sensors into neural networks. These neural networks rely on hardware computing power and are constrained by the independent nature of the sensing, storage, and computation units within the von Neumann architecture. This results in significant time and energy consumption for the neural network to complete its task. Furthermore, with the continuous development of the image sensor industry, the pixel size has increased dramatically, leading to massive amounts of image data, further highlighting the aforementioned problems.

[0004] To address these issues, scholars have conducted extensive research, primarily focusing on two aspects: one is image space classification methods, which classify images using low-level features such as texture, shape, and location. However, these methods share the drawback of not addressing the large amount of data involved and have high computational complexity. The other approach is feature space classification methods. These methods first transform the original image to a feature space using transformations such as KL transform or wavelet transform, and then extract high-level features from the feature space to achieve image classification. This approach can reduce data dimensionality and computational complexity.

[0005] However, the calculation process of transformation methods such as KL transform and wavelet transform, which are commonly used in current feature space-based classification methods, is still not simple and intuitive enough, and there is still a lot of room for optimization. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, the present invention aims to provide an image classification method based on orientation features.

[0007] The technical solution of the present invention is as follows:

[0008] An image classification method based on orientation features includes the following steps:

[0009] S1, convert the image to grayscale and reshape it to a pixel size of 225*225;

[0010] S2, use four 3*3 orientation operators to calculate the cosine similarity with the 3*3 slices in the reshaped image after step S1, and obtain four similarity matrices of size 75*75.

[0011] S3, perform single-threshold masking on each similarity matrix;

[0012] S4, perform feature fusion on the orientation feature information obtained by calculating the cosine similarity of the four channels;

[0013] S5, the fused feature maps are fed into the backbone network for classification.

[0014] Furthermore, the four 3*3 orientation operators in step S2 include black pixels and white pixels. Specifically, three black pixels are arranged along the horizontal 0° direction, the 45° direction, the vertical 90° direction, and the 135° direction, and the rest are white pixels, which are used to detect lines with orientations of 0°, 45°, 90° and 135° in the image, respectively.

[0015] Furthermore, the threshold for single-threshold masking in step S3 is in the range of 0.5 to 0.6.

[0016] Furthermore, the threshold of the single threshold masking is 0.567, that is, for each value in the calculated similarity matrix, if the value is greater than 0.567, it is set to 1, otherwise it is set to 0, so that the four similarity matrices become a 75*75 binary matrix of 0 and 1.

[0017] Furthermore, the feature fusion method in step S4 is an OR operation, that is, as long as there is a 1 at the same position of the pixel in the four similarity matrices, the value of the same position in the final fused feature map is set to 1; only when all the pixels at the same position in the four similarity matrices are 0, the value of the same position in the final fused feature map is set to 0.

[0018] Furthermore, the input layer of the backbone network in step S5 is a single-channel 75*75 data format.

[0019] The image classification method based on orientation features provided by this invention belongs to the feature space classification method. Compared with the cumbersome preliminary calculation process of existing schemes, the method of this invention can obtain a feature space that can better represent the original image with extremely simple calculations, which significantly reduces the amount of data, network size and computational overhead of the entire system from the source. At the same time, its performance in classification tasks is basically comparable to traditional methods. Attached Figure Description

[0020] Figure 1 This is a flowchart of the image classification process of the present invention;

[0021] Figure 2 This is a schematic diagram of the four 3*3 orientation operator templates used in the embodiments of the present invention. (a) is used to detect lines with an orientation of 0° in the image, (b) is used to detect lines with an orientation of 90° in the image, (c) is used to detect lines with an orientation of 45° in the image, and (d) is used to detect lines with an orientation of 135° in the image.

[0022] Figure 3 This is a concrete illustration of the image classification process of the present invention;

[0023] Figure 4 These are schematic diagrams of the four categories and corresponding classification results of the four-category experiment in this embodiment of the invention. Detailed Implementation

[0024] Figure 1 This is a flowchart of the image classification process involved in an embodiment of the present invention. For an image of arbitrary pixel size, it is first grayscaled and then reshaped to a 225*225 pixel scale using bilinear interpolation; then... Figure 2 The four 3*3 orientation operators (used to detect the direction of lines in the grayscale image) given in the paper are used to calculate the cosine similarity with 75*75 3*3 slices in the reconstructed image, resulting in four 75*75 similarity matrices. Single threshold masking is applied to each similarity matrix. Then, the orientation feature information obtained by calculating the cosine similarity of the four channels is fused. Finally, the fused feature map is fed into the backbone network for classification.

[0025] Figure 2 Four 3*3 orientation operator templates used in this embodiment are given, where black pixels represent a value of 1 and white pixels represent a value of 0, which are used to detect the orientation of 0°, 45°, 90° and 135° in the image, respectively. After calculating the cosine similarity, each 3*3 slice of the original image becomes a single value, and the original image with a pixel size of 225*225 becomes four 75*75 similarity matrices.

[0026] For different images, the optimal threshold for single threshold masking is generally 0.5 to 0.6. In this embodiment, the threshold used for single threshold masking is 0.567. That is, for each value in the calculated similarity matrix, if the value is greater than 0.567, it is set to 1, otherwise it is set to 0, so that the four similarity matrices become a 75*75 binary matrix of 0 and 1.

[0027] In this embodiment, the feature fusion method is an OR operation. That is, as long as there is a 1 at the same position of the pixel in the four similarity matrices, the value of that position in the final fused feature map is set to 1; only when all the pixels at the same position in the four similarity matrices are 0, the value of that position in the final fused feature map is set to 0.

[0028] In this embodiment, the backbone network used for classification can be any of the widely used classification neural networks. In particular, in order to adapt to the data structure in this invention, the input layer structure of the original network needs to be modified to a single-channel 75*75 data format.

[0029] Figure 3 The image classification process of this invention is illustrated more intuitively using an airplane image as an example: For an airplane image a, it is first grayscaled and then reshaped into a 225*225 pixel scale to obtain image b; then using... Figure 2 The four 3x3 orientation operators given in the example are used to calculate the cosine similarity with the 3x3 slices in the original image, resulting in four 75x75 similarity matrices. Each similarity matrix is ​​then subjected to a single threshold mask with a threshold of 0.567 to obtain Figure c. The four images from top to bottom represent the presence of lines with orientations of 0°, 45°, 90°, and 135° in each 3x3 slice of image b, with black pixels indicating their presence and white pixels indicating their absence. Subsequently, the orientation features obtained from calculating the cosine similarity of the four channels are fused to obtain image d. Black pixels represent the presence of at least one line with an orientation of 0°, 45°, 90°, or 135° in the corresponding 3x3 slice of image b, while white pixels represent the absence of such lines in the corresponding 3x3 slice of image b. The pixel grayscale values ​​are relatively evenly distributed. Finally, the fused feature map is fed into a modified ResNet18 network for classification to obtain the classification result e.

[0030] To verify the feasibility and effectiveness of the method of the present invention, a four-classification experiment was conducted in this embodiment. (Reference) Figure 4 Images of four categories of objects—airplanes, cars, cats, and horses—were collected. Each category contained 4,000 training images and 1,000 test images. After preprocessing the images into the required fusion feature maps, they were fed into a modified ResNet18 network for training and testing. The training iterations were set to 300, and the batch size to 64. The network achieved a recognition accuracy of 94.3% on the test set, which is not significantly lower than the 99% accuracy of existing methods.

[0031] Specifically, for the ResNet18 network used in this invention, its original standard input data format is a three-channel 224*224*8bit. This data is reshaped using grayscale conversion and bilinear interpolation, and then quantized from 8 bits of data per pixel to single bits of 0 and 1, ultimately transforming it into a single-channel 75*75*1bit data format, reducing the data volume by 216 times. This reduction in input layer data volume directly leads to a smaller network size and a lower computational power requirement, conservatively estimated to be 50 times lower.

Claims

1. An image classification method based on orientation features, characterized in that, Includes the following steps: S1, convert the image to grayscale and reshape it to a pixel size of 225*225; S2, use four 3*3 orientation operators to calculate the cosine similarity with the 3*3 slices in the reshaped image after step S1, and obtain four similarity matrices of size 75*75. S3, perform single-threshold masking on each similarity matrix; the threshold for single-threshold masking is 0.567, that is, for each value in the calculated similarity matrix, if the value is greater than 0.567, it is set to 1, otherwise it is set to 0, so that the four similarity matrices become 75*75 binary matrices of size 0 and 1. S4, perform feature fusion on the orientation feature information obtained by calculating cosine similarity of the four channels; wherein, the feature fusion method is OR operation, that is, as long as there is 1 in the pixels at the same position in the four similarity matrices, the value of the same position in the final fused feature map is set to 1; only when all the pixels at the same position in the four similarity matrices are 0, the value of the same position in the final fused feature map is set to 0. S5, the fused feature maps are fed into the backbone network for classification.

2. The image classification method based on orientation features according to claim 1, characterized in that, The four 3*3 orientation operators in step S2 include black pixels and white pixels. Specifically, three black pixels are arranged along the horizontal 0° direction, the 45° direction, the vertical 90° direction, and the 135° direction, and the rest are white pixels, which are used to detect lines with orientations of 0°, 45°, 90° and 135° in the image, respectively.

3. The image classification method based on orientation features according to claim 1, characterized in that, The input layer of the backbone network in step S5 is a single-channel 75*75 data format.

Images