Infrared image classification and recognition method based on deep convolutional neural network

Abstract

The invention belongs to the technical field of image recognition, and discloses an infrared image classification and recognition method based on a deep convolutional neural network, and the method comprises the following steps: S1, collecting an infrared image data set, and carrying out the preprocessing of the data; and S2, based on a Resnet34 classification model, constructing an infrared image classification and identification model based on a deep convolutional neural network, and performing model training. According to the infrared image classification and recognition method based on the deep convolutional neural network, not only is the remarkable recognition accuracy obtained in an infrared image classification task, but also compared with a traditional infrared image classification and recognition algorithm, the manual feature extraction process is avoided, the classification efficiency and accuracy are greatly improved, and the recognition accuracy is improved. And powerful support is provided for the development of the infrared image classification field.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an infrared image classification and recognition method based on a deep convolutional neural network. Background Art

[0002] In recent years, the field of ship image monitoring has attracted much attention, mainly using deep learning technology, especially convolutional neural network (CNN), to extract and classify ship image features, focusing on features such as infrared intensity, shape, structure, as well as multimodal data fusion and real-time monitoring systems. However, challenges such as complex background, target changes, big data processing and target tracking still need to be addressed to improve robustness and meet the needs of maritime safety, traffic management, etc.

[0003] In the field of infrared image processing, researchers have also made unremitting efforts to improve target detection and classification algorithms, such as convolutional neural network models suitable for infrared image classification, which also provides strong support for the progress in this field. Compared with traditional methods, convolutional neural networks (CNNs) have a series of advantages, including weight sharing, local perception, reduced number of parameters, improved computational efficiency, and direct processing of multi-dimensional image data. These characteristics make CNNs have potential application prospects in infrared image classification and target detection.

[0004] Therefore, based on the Resnet18 network structure, the present invention proposes a convolutional neural network model suitable for infrared image classification to overcome the difficulty of traditional infrared image classification and improve the detection performance. Summary of the invention

[0005] The purpose of the present invention is to provide an infrared image classification and recognition method based on a deep convolutional neural network, which significantly improves the recognition accuracy in the infrared image classification task. Compared with the traditional infrared image classification and recognition algorithm, it avoids the artificial feature extraction process, greatly improves the classification efficiency and accuracy, and provides strong support for the development of the infrared image classification field.

[0006] To achieve the above object, the present invention provides an infrared image classification and recognition method based on a deep convolutional neural network, comprising the following steps:

[0007] Step S1, collecting infrared image data sets and preprocessing the data;

[0008] Step S2: Based on the classification model of Resnet34, an infrared image classification and recognition model based on a deep convolutional neural network is constructed, and model training is performed.

[0009] Preferably, in step S1, the infrared image data set is preprocessed, and the specific process is as follows:

[0010] First, the infrared image dataset is classified and labeled, and different types of infrared images are marked as different categories;

[0011] Then, the CLAHE histogram equalization method is used to change the histogram of the image to change the grayscale of each pixel in the image.

[0012] Preferably, for a digital image with discrete gray levels, frequency is used instead of probability, then the transformation function T(r k ) is as follows:

[0013]

[0014] Among them, s k is the gray level of each pixel after equalization; T(r k ) is the transformation function; n i is the number of occurrences of gray level i; N is the total number of pixels in the image; 0≤r k ≤1,k=0,1,2,...,L-1; represents the gray level after normalization; k represents the gray level before normalization.

[0015] Preferably, in step S2, based on the classification model of Resnet34, an infrared image classification and recognition model based on a deep convolutional neural network is constructed, and the model consists of two convolutional layers, two pooling layers, two fully connected layers, and a softmax_linear output layer; wherein the fully connected layer adopts a modified linear activation function; and the softmax_linear output layer is used to linearly transform the output probability value.

[0016] Preferably, the training process of the infrared image classification and recognition model based on the deep convolutional neural network is as follows:

[0017] Step S21, first, pre-training is performed using imagenet big data based on the classification model of Resnet34;

[0018] Step S22: Secondly, adopt a learning rate reduction strategy to input all image data into the model in batches for training;

[0019] Step S23, then, use the feature selection transfer learning algorithm to find the common features between the Imagenet source data set and the target domain;

[0020] Step S24: Finally, these common features are used to assist knowledge transfer and complete the task.

[0021] Preferably, the last layer of the infrared image classification and recognition model based on the deep convolutional neural network uses a LogLog activation function, and the definition of the LogLog activation function is as follows:

[0022] f(x)=log(log(1+|x|));

[0023] Where x is the input value; |x| represents the absolute value of x, ensuring that the input value is non-negative; log(1+|x|) is the first step of taking the logarithm, compressing the larger input value to a smaller range; log(log(1+|x|)) is the second logarithm, further compressing the range of the input value;

[0024] The Log Log activation function converts each element in the input vector x to a value between 0 and 1, and ensures that the sum of all values ​​is 1, so a probability distribution is output; this allows the model to output the probability of each category in the classification task, as shown below:

[0025]

[0026] in, is the probability of the ith category, is the probability value of this category, is the likelihood of all probabilities and M is the total number of categories.

[0027] Therefore, the present invention adopts the above-mentioned infrared image classification and recognition method based on deep convolutional neural network. Deep learning simulates the human visual perception through neural network, and constructs a multi-layer network layer by layer from the bottom-level local abstract features, so that the machine can automatically learn the potential relationship within the data and finally obtain the perception of the overall target, which significantly reduces the number of parameters that need to be trained, simplifies the network model, and improves the training efficiency. In addition, the convolutional neural network performs well under large-scale samples and can extract richer features, thereby making the learned features more extensive and expressive. These factors together lead to the convolutional neural network showing excellent performance in infrared image classification and recognition tasks.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of an infrared image classification and recognition method based on a deep convolutional neural network;

[0030] Figure 2 It is a diagram of the training process of the infrared image classification and recognition model based on the deep convolutional neural network of the present invention;

[0031] Figure 3is a structural diagram of the residual block of the present invention;

[0032] Figure 4 This is the infrared image classification and recognition model diagram based on deep convolutional neural network of the present invention

[0033] Figure 5 is a custom data example diagram created by the present invention;

[0034] Figure 6 It is a curve diagram of the loss rate in the training set of the present invention;

[0035] Figure 7 It is a curve diagram of the loss rate in the test set of the present invention;

[0036] Figure 8 It is a curve change diagram of the accuracy rate in the training set of the present invention;

[0037] Fig. 9 It is a curve change diagram of the accuracy rate in the test set of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, the present invention provides an infrared image classification and recognition method based on a deep convolutional neural network, comprising the following steps:

[0040] Step S1: Collect infrared image data sets and preprocess the data.

[0041] In the real world, data is usually multi-angle, difficult to observe, and easily affected by climatic conditions. Because the database is too large and the data set often comes from multiple heterogeneous data sources, low-quality data will lead to low-quality results. Therefore, it is necessary to preprocess the infrared image data set. The specific process is as follows:

[0042] Firstly, the infrared image dataset was classified and labeled, and different types of infrared images were marked as different categories, such as cargo, ferry, tug, etc.

[0043] Then, the CLAHE histogram equalization method is used to change the grayscale of each pixel in the image by changing the histogram of the image, thereby enhancing the contrast of the ship image with a small dynamic range, thereby achieving the effect of enhancing the overall contrast of the ship image.

[0044] For digital images with discrete gray levels, frequency is used instead of probability, then the transformation function T(r k ) is as follows:

[0045]

[0046] Among them, s k is the gray level of each pixel after equalization; T(r k ) is the transformation function; n i is the number of occurrences of gray level i; N is the total number of pixels in the image; 0≤r k ≤1,k=0,1,2,...,L-1; represents the gray level after normalization; k represents the gray level before normalization. It can be seen that the gray level s of each pixel after equalization k It can be directly calculated from the histogram of the original image.

[0047] It should be noted that the s k It is also the normalized grayscale, and its value is between 0 and 1. Sometimes it is necessary to multiply it by L1 and then round it to make its grayscale range between 0 and L1, which is consistent with the original image.

[0048] Step S2: Based on the classification model of Resnet34, an infrared image classification and recognition model based on a deep convolutional neural network is constructed, and model training is performed.

[0049] The infrared image classification and recognition model based on deep convolutional neural network consists of two convolutional layers, two pooling layers, two fully connected layers (using rectified linear activation function), and a softmax_linear output layer (used for linear transformation to output probability values).

[0050] Among them, the training process of the infrared image classification and recognition model based on deep convolutional neural network is as follows:

[0051] Step S21, first, pre-training is performed using imagenet big data based on the classification model of Resnet34;

[0052] Step S22: Secondly, adopt a learning rate reduction strategy to input all image data into the model in batches for training;

[0053] Step S23, then, using a feature selection transfer learning algorithm, find out the common features of the Imagenet source data set and the target field, that is, the infrared image data set in the field of ship recognition;

[0054] Step S24, finally, use these common features to assist knowledge transfer and complete the task, such as Figure 2 shown.

[0055] Among them, the infrared image classification and recognition model based on deep convolutional neural network includes multiple residual blocks, and the SE module is inserted after each residual block, that is, after the convolution operation and the ReLU activation function are completed. It adaptively adjusts the feature response of the channel by explicitly recalibrating the channel feature response by explicitly simulating the interdependence between channels.

[0056] The main advantage of the SE module is that it adaptively learns and recalibrates the response weights of each channel of the feature map by introducing a channel attention mechanism. Specifically, the SE module first uses global average pooling to compress the spatial dimensions of the feature map, extract global context information, and generate a weight vector containing the importance of each channel through two fully connected layers. After the Sigmoid activation function, this weight vector can dynamically adjust the weight of each channel in the feature map, thereby enhancing the network's attention to channels with important features while suppressing unimportant or redundant channel features.

[0057] This adaptive calibration mechanism between channels significantly improves the feature expression ability of the network, enabling it to capture meaningful features more accurately and ultimately improve the performance of the model on various tasks. In addition, the SE module is designed to be very lightweight and only adds a small amount of computational overhead, so it can bring significant performance improvements without significantly increasing the complexity of the model.

[0058] like Figure 3 As shown in the figure, after the last convolutional layer of each residual block is combined with the skip connection, a convolutional triplet attention module (RotatetoAttend:ConvolutionalTripletAttentionModule) is inserted.

[0059] The advantage of the convolutional triple attention module is that it effectively captures and encodes dependencies across channels and spatial dimensions through an innovative three-branch structure. Specifically, the module uses rotation operations to convolve the input features in the horizontal, vertical, and depth dimensions to extract the interaction information between the dimensions, and then fuses this information into an attention weight and applies it to the original features, thereby enhancing key features and suppressing irrelevant information.

[0060] In addition, the convolutional triple attention module adopts a residual connection design to ensure the continuity of the original feature flow, so its lightweight architecture enables it to be seamlessly integrated into existing networks (such as ResNet34) without affecting the efficiency and speed of the network. Experimental results show that this module brings significant performance improvements in image classification tasks, effectively enhancing the network's expressiveness and generalization performance.

[0061] like Figure 4 As shown in the figure, the last layer of the model uses the LogLog activation function, which is a nonlinear activation function. Its main idea is to compress the range of input values ​​by taking logarithms, thereby introducing nonlinearity in the neural network.

[0062] (1) Log Log activation function definition is as follows:

[0063] f(x)=log(log(1+|x|));

[0064] Where x is the input value, which can be positive or negative. |x| represents the absolute value of x, ensuring that the input value is non-negative. log(1+|x|) is the first step of taking the logarithm, which compresses the larger input value into a smaller range. log(log(1+|x|)) is the second step of taking the logarithm, which further compresses the range of the input value.

[0065] (2) Log Log activation function has the following characteristics and advantages:

[0066] A. Compression effect: The Log Log activation function compresses a wide range of input values ​​into a smaller range of output values, similar to activation functions such as ReLU, but does not cause the "vanishing gradient" problem because taking the logarithm operation retains the gradient information.

[0067] B. Smoothness: The curve of the Log Log activation function is very smooth, especially when the input value is small, the output increases almost linearly; when the input value is large, the output gradually flattens out, thus avoiding the impact of over-amplifying the input value.

[0068] C. Symmetry between positive and negative values: Since the absolute value is taken, the Log Log activation function is symmetric for positive and negative input values, which may be more beneficial for scenarios that need to process input features with positive and negative variations.

[0069] The Log Log activation function compresses the range of input values ​​through two logarithmic transformations, ensuring stable gradient propagation in the neural network and compression and processing of a wide range of input data.

[0070] The Log Log activation function converts each element in the input vector x to a value between 0 and 1, and ensures that the sum of all values ​​is 1, so the output probability distribution can be obtained. This allows the infrared image classification and recognition model based on deep convolutional neural networks to output the probability of each category in the classification task, as shown below:

[0071]

[0072] in, is the probability of the ith category, is the probability value of this category, is the likelihood of all probabilities and M is the total number of categories.

[0073] Example

[0074] The present invention improves the ResNet model, constructs an infrared image classification and recognition model based on a deep convolutional neural network, and adopts a learning rate reduction strategy to optimize the training process. Although neural networks have potential in handling complex tasks, their performance is greatly affected by the quantity and quality of available training data. The present invention adopts data enhancement technology to increase the sample size by transforming and expanding the existing data, such as Figure 5 As shown, it includes the use of CLAHE histogram equalization algorithm and scaling methods. The following is a partial display of the data set.

[0075] The experimental environment of this embodiment includes: Windows 11 system, pytorch editor simulation software based on python programming, Facebook's deep learning framework PyTorch, and PC memory of 464GB.

[0076] Based on Python programming, following the algorithmic idea proposed in this invention, a convolutional neural network model was constructed. For the purpose of experiment, a custom data set was created, which included five different categories of infrared images, such as tug, speedboat, and sailing. Each category contained 1,400 images, totaling 7,000 images. Among them, the training set contained 5,500 samples and the test set contained 1,500 samples.

[0077] Before model training, the images were preprocessed and the size of each image was adjusted to 224×224 pixels to meet the input requirements of the convolutional neural network.

[0078] In order to effectively train the model and obtain the best parameter solution, a learning rate reduction strategy is adopted. All image data are fed into the model for training in batches, and the number of images processed in each batch is set to 32 to accelerate the gradient descent process while ensuring the best parameter update during training. This setting helps improve training efficiency and model performance.

[0079] Based on PyTorch, various visualization tools are used to monitor the model training process, which helps to gain a deeper understanding of the model's performance and can be compared with the curves on Tensorboard to ensure the consistency and reliability of the results.

[0080] A total of 5500 iterations were performed in the experiment. Figure 6 and Figure 7As shown in the figure, the accuracy continues to rise between 0 and 4500 steps, but the change is not obvious between 4500 and 5500 steps. During the entire iteration process, the accuracy curve fluctuates a bit because the small batch gradient descent method is used and the image data of each batch is different. However, the accuracy shows an upward trend. The initial fluctuations indicate that the model is learning, while the small fluctuations in the later stage indicate that the model is gradually converging. The parameters gradually approach the optimal solution, and the model successfully converges. Adding more iterations has little effect on the accuracy and loss rate, so further increasing the number of iterations will not bring significant improvements.

[0081] like Figure 8 and Fig. 9 As shown in the figure, the changing trends of accuracy and loss rate in the test set are similar to those in the training set. Considering the accuracy and loss rate of the training set and the test set, the convolutional neural network model shows good convergence. The accuracy of the training set and the test set reached 98.95% and 98.74% respectively, which verifies the effectiveness of the convolutional neural network in the multi-classification recognition task of infrared images and the important role of data enhancement methods.

[0082] This embodiment verifies the excellent performance of the proposed convolutional neural network structure in infrared image classification and recognition tasks through experimental results comparison. As shown in Table 1, compared with traditional methods, the convolutional neural network achieves a higher accuracy rate. At the same time, compared with other neural network methods, the performance of the method proposed in the present invention is quite outstanding. This successful performance is attributed to the end-to-end characteristics of the convolutional neural network, which can directly process image pixels, especially in large-scale data sets, it can extract more target features and effectively learn the essential characteristics of the data.

[0083] Table 1 Experimental results of different methods

[0084]

[0085]

[0086] The reason why convolutional neural networks have superior performance over traditional methods is that they can extract richer and more effective feature information. Deep learning builds machine learning models through multiple hidden layers, enabling them to learn more useful features, thereby improving the accuracy of classification and prediction. The high recognition accuracy of convolutional neural networks stems from their ability to effectively extract rich features, which confirms their effectiveness and excellent performance.

[0087] Therefore, the present invention adopts the above-mentioned infrared image classification and recognition method based on deep convolutional neural network. Deep learning simulates the human visual perception through neural network, and constructs a multi-layer network layer by layer from the bottom-level local abstract features, so that the machine can automatically learn the potential relationship within the data and finally obtain the perception of the overall target, which significantly reduces the number of parameters that need to be trained, simplifies the network model, and improves the training efficiency. In addition, the convolutional neural network performs well under large-scale samples and can extract richer features, thereby making the learned features more extensive and expressive. These factors together lead to the convolutional neural network showing excellent performance in infrared image classification and recognition tasks.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for infrared image classification and recognition based on deep convolutional neural network, characterized in that: The following steps are involved: Step S1, collecting infrared image data sets and preprocessing the data; Step S2: Based on the classification model of Resnet34, an infrared image classification and recognition model based on a deep convolutional neural network is constructed, and model training is performed.

2. The infrared image classification and recognition method based on deep convolutional neural network according to claim 1 is characterized in that: In step S1, the infrared image data set is preprocessed, and the specific process is as follows: First, the infrared image dataset is classified and labeled, and different types of infrared images are marked as different categories; Then, the CLAHE histogram equalization method is used to change the histogram of the image to change the grayscale of each pixel in the image.

3. The infrared image classification and recognition method based on deep convolutional neural network according to claim 2 is characterized in that: For digital images with discrete gray levels, frequency is used instead of probability, then the transformation function T(r k ) is as follows: Among them, s k is the gray level of each pixel after equalization; T(r k ) is the transformation function; n i is the number of occurrences of gray level i; N is the total number of pixels in the image; 0≤r k ≤1,k=0,1,2,...,L-1; represents the gray level after normalization; k represents the gray level before normalization.

4. The infrared image classification and recognition method based on deep convolutional neural network according to claim 1, characterized in that: In step S2, based on the classification model of Resnet34, an infrared image classification and recognition model based on a deep convolutional neural network is constructed. The model consists of two convolutional layers, two pooling layers, two fully connected layers, and a softmax_linear output layer; the fully connected layer adopts a rectified linear activation function; the softmax_linear output layer is used to linearly transform the output probability value.

5. The infrared image classification and recognition method based on deep convolutional neural network according to claim 4 is characterized in that: The training process of the infrared image classification and recognition model based on deep convolutional neural network is as follows: Step S21, first, pre-training is performed using imagenet big data based on the classification model of Resnet34; Step S22: Secondly, adopt a learning rate reduction strategy to input all image data into the model in batches for training; Step S23, then, use the feature selection transfer learning algorithm to find the common features between the Imagenet source data set and the target domain; Step S24: Finally, these common features are used to assist knowledge transfer and complete the task.

6. The infrared image classification and recognition method based on deep convolutional neural network according to claim 4 is characterized in that: The last layer of the infrared image classification and recognition model based on deep convolutional neural network uses the LogLog activation function. The definition of the Log Log activation function is as follows: f(x)=log(log(1+|x|)); Among them, x is the input value; |x| represents the absolute value of x, ensuring that the input value is non-negative; log(1+|x|) is the logarithm of the first step, which compresses the larger input value into a smaller range; log(log(1+|x|)) is the second logarithm, which further compresses the range of input values; The Log Log activation function converts each element in the input vector x to a value between 0 and 1, and ensures that the sum of all values ​​is 1, so a probability distribution is output; this allows the model to output the probability of each category in the classification task, as shown below: in, is the probability of the ith category, is the probability value of this category, is the likelihood of all probabilities and M is the total number of categories.

Images