Deep migration image classification method and system based on adaptive manifold learning

Abstract

The invention discloses a depth migration image classification method and system based on adaptive manifold learning, and relates to the technical field of image processing. The method specifically comprises the following steps: selecting source image data and target image data to be identified; the method comprises the following steps: preprocessing source image data and target image data to be identified, and respectively inputting the source image data and the target image data to be identified into a Resnet50 convolutional neural network containing a full connection layer; respectively aiming at the source image data and the target image data to be identified, adopting a self-adaptive manifold local linear representation method, and establishing the maximum mean value difference measurement of the source image data and the target image data to be identified based on manifold learning; and establishing a depth field adaptive loss function to realize data classification of the target image to be identified. According to the method, a local linear representation method is utilized, the distribution difference between the source image of the maximum mean value difference based on adaptive manifold learning and the target image to be recognized is established, the Resnet50 convolutional neural network comprising a full connection layer is trained, and the classification performance of the target image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a deep migration image classification method and system based on adaptive manifold learning. Background Art

[0002] In today's digital age, images, as an important information carrier, are widely used in various fields, such as medicine, security, transportation, entertainment, etc. Accurately classifying images is crucial to achieve intelligent information processing and decision-making. However, image classification faces many challenges.

[0003] On the one hand, image data is characterized by high dimensionality, complexity, and diversity. Different types of images may have huge differences in color, texture, shape, lighting, etc., which makes it difficult to extract effective features. On the other hand, practical applications often face problems such as data imbalance, high annotation costs, and changes in data distribution. For example, in some specific fields, it may be very expensive and time-consuming to obtain a large amount of annotated image data, while unlabeled data is relatively easy to obtain. In addition, the distribution of image data collected in different scenarios may be different, which makes it difficult to directly apply a classification model trained on one dataset to another dataset.

[0004] Traditional image classification methods mainly rely on manually designed feature extractors and machine learning algorithms. For example, feature extraction methods such as scale-invariant feature transform (SIFT) and histogram of oriented gradients (HOG) can extract local features of images to a certain extent, but these methods often require professional knowledge and a lot of parameter adjustment work, and have limited robustness to factors such as complex image scenes and changing lighting conditions.

[0005] At the same time, traditional machine learning algorithms such as support vector machines (SVM) and random forests may face problems such as high computational complexity and insufficient generalization ability when processing large-scale image data. Moreover, these methods usually assume that the training data and test data come from the same distribution. When the data distribution changes, the performance of the model will drop significantly.

[0006] With the development of deep learning technology, deep neural networks have achieved great success in image classification tasks. Deep neural networks can automatically learn the feature representation of images without manually designing complex feature extractors. For example, convolutional neural networks (CNNs) can effectively extract local and global features of images through the combination of convolutional layers and pooling layers, thereby greatly improving the accuracy of image classification.

[0007] However, deep neural networks also have some problems. First, training deep neural networks requires a large amount of labeled data, but in practical applications, it is often difficult to obtain a large amount of labeled data. Second, when there is a large difference in data distribution between the source domain and the target domain, the performance of the deep neural network trained directly on the source domain on the target domain will be greatly affected.

[0008] In order to solve the above problems, the present invention proposes a deep migration image classification method and system based on adaptive manifold learning. Summary of the invention

[0009] The purpose of the present invention is to propose a deep migration image classification method and system based on adaptive manifold learning to solve the problems raised in the background technology:

[0010] Traditional methods rely on manually designed feature extractors and machine learning algorithms, which have limited robustness and are computationally complex. Although deep neural networks have been successful in image classification, they require a large amount of labeled data, and their performance degrades when there is a large difference in the distribution of data between the source domain and the target domain. In addition, practical applications face challenges such as data imbalance, high labeling costs, and changes in data distribution, making it difficult to meet the needs of image classification in complex scenarios.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] A deep migration image classification method based on adaptive manifold learning includes the following steps:

[0013] S1, selecting source image data and target image data to be identified;

[0014] S2, preprocessing the selected source image data and the target image data to be identified, and inputting them into a Resnet50 convolutional neural network containing a fully connected layer respectively;

[0015] S3, respectively for the source image data and the target image data to be identified, using a local linear representation method, and establishing a distribution difference measurement of the source image data and the target image data to be identified based on manifold learning;

[0016] S4. Establish a deep domain adaptation loss function to achieve the classification of target image data to be identified.

[0017] Preferably, the preprocessing operation in S2 includes: resizing, grayscale processing, image enhancement, image denoising and the following normalization processing:

[0018]

[0019] Among them, μ represents the mean of the image grayscale vector; σ represents the standard deviation of the image grayscale vector; x represents the grayscale value of the image vector; x* Represents the normalized result;

[0020] The normalized source image data and the target image data to be recognized are used as the input of a source ResNet50 network containing a fully connected layer and a target ResNet50 network containing a fully connected layer, respectively. The source image dataset and the target image dataset are divided into a training set and a test set in a ratio of 7:3 and put into the corresponding data processing process respectively.

[0021] Preferably, the step of selecting neighboring points in S3 is specifically as follows:

[0022] S3.1: Given the initial number of neighbor points K0;

[0023] S3.2: Find image data x i The k nearest neighbor images in the nearest neighbor feature space are obtained by the function δ ik Reconstruction error:

[0024]

[0025] Calculate the local linear representation coefficient w ij as follows:

[0026]

[0027] G ik =(x i -x ig )(x i -x ik )

[0028] Among them, x ij Represents the distance x in the image data i The jth image that is closest and constitutes the nearest neighbor feature space; G ik represents the local Grammar matrix of image data; x ig and x ik All of them are composed of x i An image of the nearest neighbor feature space;

[0029] S3.3: For any image x i , k is traversed from 2 to K0 with a step size of 1 as the number of neighboring points, where K0 is a natural number greater than 2, and the neighboring image is selected to obtain the reconstruction error sequence

[0030] S3.4: From the reconstruction error sequence Select the K corresponding to the smallest error value as the number of adaptive neighbor points, and select the corresponding local linear representation coefficient w ij(j=2,...,K) is the adaptive local linear representation coefficient.

[0031] Preferably, in S3, for the source image data and the target image data to be identified, S3.1 to S3.4 are repeated to respectively find the source image data and the target image data to be identified. The number of adaptive neighbor images K in the nearest neighbor feature space s and the corresponding adaptive local linear representation coefficients And the target image data to be identified The number of adaptive neighbor images K in the nearest neighbor feature space t and the corresponding adaptive local linear representation coefficients Linearly represent the source image data and the target image data to be identified respectively, so as to minimize the linear representation error;

[0032] Then the maximum mean difference based on local linear representation is used as the distribution difference L between the source image data and the target image data to be identified. LLE , as follows:

[0033]

[0034] Among them, n s Indicates the number of all source images; n t represents the number of target images to be identified; H represents the reproducing kernel Hilbert space; Represents the distance source image data The mth image that is closest to and constitutes the nearest neighbor feature space Gaussian kernel transform of ; Indicates the distance to be identified target image data The nth image that is closest and constitutes the nearest neighbor feature space Gaussian kernel transform of .

[0035] Preferably, the S4 is specifically as follows:

[0036] The maximum mean difference based on local linear representation is used as the loss function of ResNet50 network transfer learning containing a fully connected layer, and the cross entropy is used as the classification loss function of the target image to be identified. Therefore, the total deep learning loss function L total It is expressed as follows:

[0037] L total =L Cross-Entroy +βL LLE

[0038] Among them, L Cross-Entroy represents the cross entropy loss; β is a constant.

[0039] A deep migration image classification system based on adaptive manifold learning, comprising:

[0040] Image data selection module: used to select source image data and target image data to be identified;

[0041] Image data preprocessing module: used to preprocess the source image data and the target image data to be identified, and input them into the ResNet50 deep convolutional neural network containing a fully connected layer respectively;

[0042] Distribution difference measurement establishment module: for the source image data and the target image data to be identified, a manifold local linear representation is used;

[0043] Image data classification module to be identified: establish a deep domain adaptation loss function to achieve the classification of target image data to be identified.

[0044] Compared with the prior art, the present invention provides a deep migration image classification method and system based on adaptive manifold learning, which has the following beneficial effects:

[0045] The present invention captures the local structure of image data through a local linear representation method, and also establishes a distribution difference metric to help quantify the difference between source image data and target image data, which can improve the model's ability to migrate between different data sets; the present invention combines deep neural networks with manifold learning to improve classification accuracy, can adapt to complex scenes, can reduce the impact of image resolution and device differences, and make the model more stable and universal; it can also reduce annotation costs and use source domain knowledge to migrate to the target domain. It also promotes cross-domain applications and promotes technical exchanges and knowledge reuse in different fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the method mentioned in Example 1 of the present invention;

[0047] Figure 2 This is a diagram of the deep migration network architecture mentioned in Example 1 of the present invention;

[0048] Figure 3 This is a system block diagram mentioned in Example 2 of the present invention.

[0049] Meaning of the symbols in the figure:

[0050] 11. Image data selection module; 12. Image data preprocessing module; 13. Distribution difference measurement establishment module; 14. Image data classification module to be identified. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0052] The present invention captures the local structure of image data through a local linear representation method, and also establishes a distribution difference metric to help quantify the difference between source image data and target image data, which can improve the model's ability to migrate between different data sets; the present invention combines deep neural networks with manifold learning to improve classification accuracy, can adapt to complex scenes, and can reduce the impact of image resolution and device differences, making the model more stable and universal; it can also reduce annotation costs and use source domain knowledge to migrate to the target domain. It also promotes cross-domain applications and promotes technical exchanges and knowledge reuse in different fields. Specifically includes the following content.

[0053] Embodiment 1:

[0054] See also Figure 1-2 The present invention provides a deep migration image classification method based on adaptive manifold learning, comprising the following steps:

[0055] S1. Select source image data and target image data to be identified; the details are as follows:

[0056] A large labeled image dataset is used as the source image data, and a set of newly collected bit-labeled images are used as the target image data to be identified.

[0057] From the public image database, we selected images of 100 different kinds of objects, with about 40 to 258 images of each object, totaling more than 8,000 images. These images have different lighting conditions, angles, backgrounds, etc., which can well represent various actual shooting scenes.

[0058] A digital camera was used to collect new images of the above objects as target image data to be identified. In order to cover as many types of objects and shooting conditions as possible, the images were taken at different times, weather and lighting conditions. Finally, about 500 unlabeled object images were obtained, which had different resolutions, angles and backgrounds.

[0059] S2. Preprocess the selected source image data and the target image data to be identified, and input them into a Resnet50 convolutional neural network containing a fully connected layer respectively; the details are as follows:

[0060] The source image data collected in S1 and the target image data to be identified are sequentially subjected to resizing, grayscale processing, image enhancement, image denoising and the following normalization processing:

[0061]

[0062] Among them, μ represents the mean of the image grayscale vector; σ represents the standard deviation of the image grayscale vector; x represents the grayscale value of the image vector; x * Represents the normalized result;

[0063] Reference Figure 2 Finally, the normalized source image data and the target image data to be recognized are used as the input of the source ResNet50 network containing a fully connected layer and the target ResNet50 network containing a fully connected layer respectively; the source image dataset and the target image dataset are divided into training set and test set in a ratio of 7:3 and put into the corresponding data processing process respectively.

[0064] S3. For the source image data and the target image data to be identified, respectively, a local linear representation method is used, and a distribution difference measurement of the source image data and the target image data to be identified based on manifold learning is established; the details are as follows:

[0065] The steps for selecting neighboring points are as follows:

[0066] S3.1: Given the initial number of neighbor points K0;

[0067] S3.2: Find image data x i The k nearest neighbor images in the nearest neighbor feature space are obtained by the function δ ik Reconstruction error:

[0068]

[0069] Calculate the local linear representation coefficient w ij as follows:

[0070]

[0071] G ik =(x i -x ig )(x i -x ik )

[0072] Among them, x ij Represents the distance x in the image data i The jth image that is closest and constitutes the nearest neighbor feature space; G ik represents the local Grammar matrix of image data; x ig and x ik All of them are composed of x i An image of the nearest neighbor feature space;

[0073] S3.3: For any image x i, k is traversed from 2 to K0 with a step size of 1 as the number of neighboring points, where K0 is a natural number greater than 2, and the neighboring image is selected to obtain the reconstruction error sequence

[0074] S3.4: From the reconstruction error sequence Select the K corresponding to the smallest error value as the number of adaptive neighbor points, and select the corresponding local linear representation coefficient w ij (j=2,...,K) is the adaptive local linear representation coefficient.

[0075] For the source image data and the target image data to be identified, repeat S3.1 to S3.4 to find the components of the source image data. The number of adaptive neighbor images K in the nearest neighbor feature space s and the corresponding adaptive local linear representation coefficients And the target image data to be identified The number of adaptive neighbor images K in the nearest neighbor feature space t and the corresponding adaptive local linear representation coefficients Linearly represent the source image data and the target image data to be identified respectively, so as to minimize the linear representation error;

[0076] For all sample points in the source image data set and all sample points in the target image data set, the maximum mean difference based on local linear representation is used as the distribution difference L between the source image data and the target image data to be identified. LLE , as follows:

[0077]

[0078] Among them, n s Indicates the number of all source images; n t represents the number of target images to be identified; H represents the reproducing kernel Hilbert space; Represents the distance source image data The mth image that is closest to and constitutes the nearest neighbor feature space Gaussian kernel transform of ; Indicates the distance to be identified target image data The nth image that is closest and constitutes the nearest neighbor feature space Gaussian kernel transform of .

[0079] S4. Establish a deep domain adaptation loss function to achieve the classification of target image data to be identified. The details are as follows:

[0080] The maximum mean difference based on local linear representation is used as the loss function of ResNet50 network transfer learning containing a fully connected layer, and the cross entropy is used as the classification loss function of the target image to be identified. Therefore, the total deep learning loss function L total It is expressed as follows:

[0081] L total =L Cross-Entroy +βL LLE

[0082] Among them, L Cross-Entroy represents the cross entropy loss; β is a constant.

[0083] The source image dataset and the target image dataset are divided into a training set and a validation set according to a certain ratio. Using the pre-trained model as the base model, the model parameters can be initialized to the parameters of the pre-trained model. For each iteration, a batch of samples are randomly selected from the source image training set and the distribution difference loss L is calculated. LLE , and randomly select a batch of samples from the target image training set and calculate the cross entropy loss L Cross-Entroy .

[0084] Calculate the total loss, and back-propagate and optimize the model parameters according to the loss function, such as using stochastic gradient descent (SGD) or Adam optimization algorithm. Regularly evaluate the performance of the model on the validation set, and adjust the hyperparameters (such as learning rate, etc.) and model structure based on the validation results. When the model training is completed, use the trained model to classify and predict the images in the target image test set. For the target image, the model outputs the corresponding predicted category label.

[0085] Embodiment 2:

[0086] See also Figure 3 The present invention provides a deep migration image classification system based on adaptive manifold learning, comprising:

[0087] Image data selection module 11: used for selecting source image data and target image data to be identified;

[0088] Image data preprocessing module 12: used to preprocess the source image data and the target image data to be identified, and input them into a ResNet50 deep convolutional neural network including a fully connected layer respectively;

[0089] A distribution difference measurement establishment module 13: for the source image data and the target image data to be identified, a manifold local linear representation is adopted;

[0090] The module 14 for classifying the image data to be identified is to establish a deep domain adaptation loss function to classify the target image data to be identified.

[0091] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A deep migration image classification method based on adaptive manifold learning, characterized in that: The steps include: S1, selecting source image data and target image data to be identified; S2, preprocessing the selected source image data and the target image data to be identified, and inputting them into a Resnet50 convolutional neural network containing a fully connected layer respectively; S3, adaptively selecting neighboring points for each image in the selected source image data and the target image data to be identified; S4, respectively, for the source image data and the target image data to be identified, a local linear representation method is used, and a distribution difference measurement of the source image data and the target image data to be identified is established based on manifold learning; S5. Establish a deep domain adaptation loss function to achieve classification of target image data to be identified.

2. The method for deep migration image classification based on adaptive manifold learning according to claim 1, characterized in that: The preprocessing operations in S2 include: resizing, grayscale processing, image enhancement, image denoising and the following normalization processing: Among them, μ represents the mean of the image grayscale vector; σ represents the standard deviation of the image grayscale vector; x represents the grayscale value of the image vector; x * Represents the normalized result; The normalized source image data and the target image data to be recognized are used as the input of a source ResNet50 network containing a fully connected layer and a target ResNet50 network containing a fully connected layer, respectively. The source image dataset and the target image dataset are divided into a training set and a test set in a ratio of 7:3 and put into the corresponding data processing process respectively.

3. The method for deep migration image classification based on adaptive manifold learning according to claim 1, characterized in that: The steps of selecting the neighboring points in S3 are as follows: S3.1: Given the initial number of neighbor points K0; S3.2: Find image data x i The k nearest neighbor images in the nearest neighbor feature space are obtained by the function δ ik Reconstruction error: Calculate the local linear representation coefficient w ij as follows: Among them, x ij Represents the distance x in the image data i The jth image that is closest and constitutes the nearest neighbor feature space; G ik represents the local Grammar matrix of image data; x ig and x ik All of them are composed of x i An image of the nearest neighbor feature space; S3.3: For any image x i , k is traversed from 2 to K0 with a step size of 1 as the number of neighboring points, where K0 is a natural number greater than 2, and the neighboring image is selected to obtain the reconstruction error sequence S3.4: From the reconstruction error sequence Select the K corresponding to the smallest error value as the number of adaptive neighbor points, and select the corresponding local linear representation coefficient w ij (j=2,...,K) is the adaptive local linear representation coefficient.

4. The method for deep migration image classification based on adaptive manifold learning according to claim 3, characterized in that: In S3, for the source image data and the target image data to be identified, S3.1 to S3.4 are repeated to respectively find the components of the source image data The number of adaptive neighbor images K in the nearest neighbor feature space s and the corresponding adaptive local linear representation coefficients And the target image data to be identified The number of adaptive neighbor images K in the nearest neighbor feature space t and the corresponding adaptive local linear representation coefficients Linearly represent the source image data and the target image data to be identified respectively, so as to minimize the linear representation error; Then the maximum mean difference based on local linear representation is used as the distribution difference L between the source image data and the target image data to be identified. LLE , as follows: Among them, n s Indicates the number of all source images; n t represents the number of target images to be identified; H represents the reproducing kernel Hilbert space; Represents the distance source image data The mth image that is closest to and constitutes the nearest neighbor feature space Gaussian kernel transform of ; Indicates the distance to be identified target image data The nth image that is closest and constitutes the nearest neighbor feature space Gaussian kernel transform of .

5. The method for deep migration image classification based on adaptive manifold learning according to claim 4, characterized in that: The S4 is specifically as follows: The maximum mean difference based on local linear representation is used as the loss function of ResNet50 network transfer learning containing a fully connected layer, and the cross entropy is used as the classification loss function of the target image to be identified. Therefore, the total deep learning loss function L total It is expressed as follows: L total =L Cross-Entroy +βL LLE Among them, L Cross-Entroy represents the cross entropy loss; β is a constant.

6. A deep migration image classification system based on adaptive manifold learning as claimed in any one of claims 1 to 5, characterized in that: include: Image data selection module (11): used for selecting source image data and target image data to be identified; Image data preprocessing module (12): used to preprocess source image data and target image data to be identified, and input them into a ResNet50 deep convolutional neural network including a fully connected layer respectively; Distribution difference measurement establishment module (13): for source image data and target image data to be identified, a manifold local linear representation is used; The module for classifying the image data to be identified (14) is to establish a deep domain adaptation loss function to classify the target image data to be identified.

Images