Low-resolution industrial image classification method and device based on label-guided diversity contrast learning

Abstract

The invention belongs to the related technical field of image classification, and discloses a low-resolution industrial image classification method and equipment based on label-guided diversity contrast learning, and the method comprises the steps: (1) training a feature extractor in a neural network model through employing low-resolution industrial image data with category labels and a diversity contrast learning strategy, extracting a representation vector of a low-resolution image of a known category through a feature extractor, and using the representation vector to train a classifier in a neural network model to obtain a low-resolution image classification model; a sequence decoupling large kernel convolution module is embedded in the feature extractor; and (2) inputting low-resolution images to be classified into the obtained low-resolution image classification model, extracting features from the corresponding low-resolution images by a feature extractor and generating representation vectors, decoding the representation vectors by a classifier and outputting classification results, and realizing automatic classification of the low-resolution images. According to the invention, the classification accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to image classification, and more specifically, relates to a low-resolution industrial image classification method and device based on label-guided diversity contrast learning. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, industrial image acquisition and analysis technologies have been widely used in production monitoring, quality inspection, equipment maintenance, and other fields. However, in actual industrial production processes, due to limitations in equipment, environment, and operating conditions, the resolution of acquired images is often low. This low-resolution imagery can lead to a loss of image detail and is often accompanied by noise, blur, and other issues, which in turn hinders downstream analysis and processing.

[0003] In industrial visual inspection, the occurrence of low-resolution images is common and unavoidable. For example, in production line monitoring, due to the need to capture a large number of images in real time, image resolution is often compressed to a low level due to limitations in bandwidth, equipment processing capabilities, and real-time requirements. In extreme environments such as high temperature, high pressure, and radiation, to avoid damage to equipment or operators, images are usually acquired from a distance or using sensors made of special materials, resulting in low image resolution and blurred details. Furthermore, in large-scale production, to reduce hardware costs and improve data processing efficiency, large-area images captured by mid- to low-end cameras are often used for rapid pre-screening.

[0004] Image classification is a key task in industrial visual inspection, designed to help automated systems make decisions by effectively classifying and identifying images. Image classification technology can solve many practical problems in industrial production. For example, in manufacturing, image classification technology can monitor the status of the production line in real time, promptly identify problems and issue corresponding alarms, prompting engineers to make process adjustments, thereby improving product consistency and quality. In factory quality inspection, image classification technology can distinguish between different categories of product images, automatically separating qualified from unqualified products, reducing manual intervention and improving production efficiency. In equipment maintenance, image classification technology can detect minor damage or wear on equipment surfaces, providing early warning and avoiding major failures and equipment downtime.

[0005] Low-resolution image classification plays a significant role in improving production efficiency, reducing costs, and enhancing product quality. Because low-resolution images require less storage, transmission, and processing, using them for classification and inspection effectively reduces reliance on high-end image acquisition equipment, lowering system construction and operating costs. Furthermore, low-resolution image classification technology significantly improves image processing efficiency, making it particularly suitable for rapid defect screening and real-time monitoring in large-scale production. Therefore, low-resolution image classification technology holds strategic importance for promoting intelligent and automated industrial production.

[0006] In summary, low-resolution image classification technology has important engineering significance in the field of industrial automation. However, existing image classification methods are mostly designed for high-resolution images and cannot effectively address the challenges posed by low-resolution images. Classification accuracy in low-resolution scenes will be significantly reduced. Summary of the Invention

[0007] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a low-resolution industrial image classification method and device based on label-guided diversity contrastive learning, which aims to solve the problem of low classification accuracy of existing low-resolution images.

[0008] To achieve the above objectives, according to one aspect of the present invention, a low-resolution industrial image classification method based on label-guided diversity contrastive learning is provided. The classification method comprises the following steps:

[0009] (1) Using pre-acquired low-resolution industrial image data with category labels and a diversity contrast learning strategy to train a feature extractor in a neural network model, extracting representation vectors of low-resolution images of known categories through the trained feature extractor, and using the extracted representation vectors to train a classifier in the neural network model, thereby obtaining a low-resolution image classification model; wherein the feature extractor is embedded with a sequence decoupled large kernel convolution module;

[0010] (2) The low-resolution image to be classified is input into the obtained low-resolution image classification model, the feature extractor extracts features from the corresponding low-resolution image and generates a representation vector, the classifier decodes the representation vector and outputs the classification result, thereby realizing automatic classification of the low-resolution image.

[0011] Furthermore, the feature extractor uses a convolutional neural network structure, and the classifier uses a feedforward neural network structure, which are used to decode the representation vector from the feature extractor layer by layer, thereby achieving classification of the low-resolution image.

[0012] Furthermore, the feature extractor uses ResNet18 as the backbone, and the classifier is divided into three layers: input layer, hidden layer, and output layer.

[0013] Furthermore, the sequence decoupling large kernel convolution module includes two parts: sequence decoupling and sequence space selection. Sequence decoupling uses depth-wise convolution sequence Explicitly decouple the large kernel convolution mechanism, then use Ghost convolution Restore the channel suppression of each layer of depth-wise convolution process; Sequence space selection first splices the feature map H extracted from all scale receptive fields along the channel dimension i And through parallel channels, the feature maps extracted from all scale receptive fields are spliced along the channel dimension to obtain the overall feature map, which is subjected to global average pooling and global maximum pooling. Get 2 pixel-level pooling feature matrices; then, use depooling convolution The number of channels is restored to the sequence value N, and the sigmoid function is used to activate the feature map restored by the de-pooling convolution to obtain the mask weight of each large kernel decomposition sequence; finally, the de-pooling convolution is used After restoring the number of input channels, the input X connected with the residual i Make Hadamard product and get the final output value Y i .

[0014] Furthermore, the final output value Y i The formula is:

[0015]

[0016] The classifier contains a dropout layer to impose regularization constraints.

[0017] Furthermore, the input of the feature extractor is the original low-resolution image x i and its corresponding category label y i , the output is a representation vector containing discriminative information The parameters θ of the neural network are optimized by minimizing the distance between the representation vectors of similar images and maximizing the distance between the representation vectors of heterogeneous images.

[0018] Furthermore, the label-guided diversity contrastive learning strategy is divided into two parts: label-guided contrastive learning and diversity contrastive learning; label-guided contrastive learning learns a low-dimensional representation by maximizing the consistency of positive sample pairs and minimizing the consistency of negative sample pairs. Assume data set There are n samples in total Then the loss function of label-guided contrastive learning is:

[0019]

[0020] in, and Represent the positive sample pair set and the negative sample pair set respectively, x P is a positive sample pair of sample x, x N is a negative sample pair of sample x, and f(·;θ) is the mapping relationship learned by the neural network.

[0021] Furthermore, diversity contrastive learning introduces an additional quadratic regularization term based on the Gram matrix in the loss function of label-guided contrastive learning. To penalize similarity and promote the orthogonalization of the weight matrix W to learn more diverse representations, the loss function of label-guided diversity contrastive learning is:

[0022]

[0023] The present invention also provides a low-resolution industrial image classification system based on label-guided diversity contrast learning, the system including a memory and a processor, the memory storing a computer program, and the processor executing the low-resolution industrial image classification method based on label-guided diversity contrast learning as described above when executing the computer program.

[0024] The present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the low-resolution industrial image classification method based on label-guided diversity contrast learning as described above.

[0025] In general, compared with the prior art, the low-resolution industrial image classification method and device based on label-guided diversity contrastive learning provided by the present invention has the following beneficial effects:

[0026] 1. This paper constructs a neural network model LR-CLS based on a feature extractor-classifier framework and introduces a sequence decoupled large kernel convolution module and a label-guided diversity contrastive learning strategy to achieve efficient and accurate classification of low-resolution industrial images. It overcomes the dependence of traditional image classification methods on high-resolution images, reduces the computational cost of the data processing process, and improves the accuracy of low-resolution image classification.

[0027] 2. This paper proposes a sequential decoupled large kernel convolution module, which uses a depth-wise convolution sequence to independently extract spatial features and restores channel suppression through a ghost module, explicitly decoupling the classic large kernel convolution; then, through a sequential spatial selection mechanism, the optimal filter is dynamically selected according to the foreground size of the input image, achieving low-cost and adaptive large-scale feature interaction.

[0028] 3. The present invention adopts a label-guided diversity contrastive learning strategy, which uses prior knowledge to optimize the training process of classical self-supervised contrastive learning, and promotes the orthogonalization of the weight matrix through a quadratic regularization term based on the Gram matrix to obtain a diverse representation, thereby improving the model's ability to extract discriminative features under low-resolution conditions.

[0029] 4. Diversity Contrastive Learning In the loss function of label-guided contrastive learning, an additional quadratic regularization term based on the Gram matrix is introduced. To penalize similarity, promote the orthogonalization of the weight matrix W to learn more diverse representations, thereby improving the model's feature extraction ability for low-resolution images.

[0030] 5. Sequence decoupling using depth-wise convolution sequence Explicitly decouple the large kernel convolution mechanism, then use Ghost convolution Restore the channel suppression of each layer of depth-wise convolution process and realize the input feature map Low-cost, large-scale information interaction.

[0031] 6. The classifier includes a dropout layer to impose regularization constraints and avoid overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a low-resolution industrial image classification method based on label-guided diversity contrastive learning provided by the present invention;

[0033] Figure 2 This is a diagram of the LR-CLS network structure constructed in an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of the label-guided diversity comparative learning strategy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0036] The present invention provides a low-resolution industrial image classification method based on label-guided diversity contrastive learning. The classification method achieves efficient and accurate classification of low-resolution industrial images by constructing a neural network model with a feature extractor-classifier framework and introducing a sequence decoupling large kernel convolution module and a label-guided diversity contrastive learning strategy.

[0037] This paper develops a low-resolution image classification model based on deep learning technology, achieving efficient and automatic classification of low-resolution images. This method has low cost and high efficiency in the signal analysis process, while also performing well in classification accuracy.

[0038] See also Figure 1 , the classification method mainly includes the following steps:

[0039] S1 builds a deep learning-based neural network model, LR-CLS, which uses a feature extractor-classifier framework. The feature extractor, using a convolutional neural network (CNN) architecture, extracts effective features from low-resolution images and converts them into high-order representation vectors containing discriminative information. The classifier, using a feedforward neural network (FFN) architecture, decodes the representation vector layer by layer, thereby accurately classifying low-resolution images.

[0040] like Figure 2 As shown in the figure, the feature extractor uses ResNet18 as the backbone and embeds a sequence decoupled large kernel convolution module; the classifier uses the FFN structure and is divided into three layers: input layer, hidden layer and output layer.

[0041] The feature extractor uses a CNN with residual connections as its backbone and embeds a sequence-decoupled large-kernel convolution module. This enables the feature extractor to dynamically and adaptively select the feature map most suitable for the detected object size from large kernels of different scales. Specifically, the sequence-decoupled large-kernel convolution module consists of two parts: sequence decoupling and sequence space selection.

[0042] Sequence decoupling using depth-wise convolution sequence Explicitly decouple the large kernel convolution mechanism, then use Ghost convolution Restore the channel suppression of each layer of depth-wise convolution process and realize the input feature map Low-cost, large-scale information interaction:

[0043]

[0044] Sequence space selection first concatenates the feature maps H extracted from all scale receptive fields along the channel dimension. iAnd through parallel channels, the feature maps extracted from all scale receptive fields are spliced along the channel dimension to obtain the overall feature map, which is subjected to global average pooling and global maximum pooling. Get 2 pixel-level pooling feature matrices. Then, use the depooling convolution The number of channels is restored to the sequence value N, and the sigmoid function is used to activate the feature map restored by the de-pooling convolution to obtain the mask weight of each large kernel decomposition sequence. Finally, the de-pooling convolution is used After restoring the number of input channels, the input X connected with the residual i Make Hadamard product and get the final output value Y i :

[0045]

[0046] The classifier includes a dropout layer to impose regularization constraints and avoid overfitting. The dimension of the representation vector output by the feature extractor is positively correlated with the resolution of the input image.

[0047] S2, using a pre-acquired low-resolution industrial image dataset with clear category labels Train the feature extractor in the neural network model. Specifically, the input of the feature extractor is the original low-resolution image x i and its corresponding category label y i , the output is a high-order representation vector containing discriminative information The training process adopts the contrastive learning method to optimize the parameters θ of the neural network by minimizing the distance between the representation vectors of similar images and maximizing the distance between the representation vectors of heterogeneous images.

[0048] In this embodiment, the input is an image x with a resolution of 32×32. i and its category label y i , the output is a 128-dimensional representation vector Cosine similarity is used to quantify the distance between different image representation vectors. Training is performed for 1000 epochs, with an initial learning rate of 0.5, which is decayed by 50% at epochs 500 and 850, respectively. The loss function is a label-guided diversity contrastive learning strategy, and the model parameters θ are optimized epoch by epoch using the SGD optimizer.

[0049] The schematic diagram of the label-guided diversity contrast learning strategy is as follows Figure 3 As shown in Figure 2, it is divided into two parts: label-guided contrastive learning and diversity contrastive learning. Label-guided contrastive learning learns a low-dimensional representation by maximizing the consistency of positive sample pairs and minimizing the consistency of negative sample pairs. Hypothetical dataset There are n samples in total Then the loss function of label-guided contrastive learning is:

[0050]

[0051] in, and Represent the positive sample pair set and the negative sample pair set respectively, x P is a positive sample pair of sample x, x N is a negative sample pair of sample x, and f(·;θ) is the mapping relationship learned by the neural network.

[0052] Diversity Contrastive Learning introduces an additional quadratic regularization term based on the Gram matrix into the loss function of label-guided contrastive learning. To penalize similarity, promote the orthogonalization of the weight matrix W to learn more diverse representations, thereby improving the model's feature extraction ability for low-resolution images. Therefore, the loss function of label-guided diversity contrastive learning is:

[0053]

[0054] During the training process of the feature extractor, cosine similarity is selected to quantify the distance between different image representation vectors, and the SGD optimizer is selected to optimize the model parameters of the feature extractor generation by generation.

[0055] S3, extract the representation vector of the low-resolution image of known category through the trained feature extractor, and use the extracted representation vector to train the classifier in the neural network. Specifically, the input of the classifier is the representation vector of the low-resolution image and its category label y i , the output is the predicted category of the image During training, the cross entropy loss strategy is used to minimize the predicted category and the true label y i The difference between them is used to optimize the parameters θ of the neural network.

[0056] During the classifier training process, the Adam optimizer is selected to optimize the model parameters of the classifier generation by generation.

[0057] In this embodiment, the input is a 128-dimensional representation vector and its category label y i , the output is the predicted category of the corresponding image The input layer has a dimension of 128, the hidden layer has a dimension of 64, and the output layer has a dimension equal to the number of classes. The first two layers are trained with a dropout coefficient of 0.2, 100 epochs, and a learning rate of 0.001. The cross-entropy loss function is used, and the model parameter θ is optimized epoch by epoch using the Adam optimizer. The cross-entropy loss is defined as:

[0058]

[0059] S4, in the feature extractor part of the neural network model framework constructed by S1, calls the model parameters of the feature extractor trained by S2, and in the classifier part, calls the model parameters of the classifier trained by S3 to obtain a complete low-resolution image classification model LR-CLS.

[0060] S5, inputs the low-resolution image to be classified into the low-resolution image classification model obtained in S4, the feature extractor extracts features from the low-resolution image and generates a representation vector, the classifier decodes the representation vector and outputs the classification result, thereby realizing automatic classification of the low-resolution image.

[0061] During the classification process, the resolution of the image to be classified should be consistent with the resolution of the image used for model training.

[0062] The present invention is further described in detail below with reference to specific embodiments.

[0063] In order to verify the practical application effect of the present invention, the present invention is verified on the Northeastern University of China hot-rolled steel surface defect dataset (NEU-CLS) and the Chongqing University of China weld surface defect dataset (RSW-C). The detailed information of the two datasets is summarized in Table 1.

[0064] Table 1 Dataset sample information

[0065]

[0066]

[0067] We selected the state-of-the-art MFN-ResNet50 model from the original paper on the NEU-CLS dataset and the state-of-the-art FCA-ResNet50 model from the original paper on the RSW-C dataset as control groups to fully verify the effectiveness of our proposed method. The input image resolution was 32×32. Each experiment was independently trained five times, and the average of the five results was used as the final evaluation metric. The experimental results are summarized in Table 2.

[0068] Table 2 Low-resolution image classification results

[0069]

[0070] As can be clearly seen in Table 2, compared with other image classification methods, the LR-CLS model provided by the present invention performs better in low-resolution image classification tasks, with accuracy exceeding the SOTA model by 1.33% and 8.46% on the NEU-CLS and RSW-C datasets, respectively. Therefore, the proposed method can effectively improve the classification accuracy of low-resolution images. Furthermore, the proposed method is applicable to a wider range of industrial scenarios and can reduce the hardware cost of industrial image acquisition and the computational cost of signal analysis.

[0071] The present invention also provides a low-resolution industrial image classification system based on label-guided diversity contrast learning, the system including a memory and a processor, the memory storing a computer program, and the processor executing the low-resolution industrial image classification method based on label-guided diversity contrast learning as described above when executing the computer program.

[0072] The present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the low-resolution industrial image classification method based on label-guided diversity contrast learning as described above.

[0073] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A low-resolution industrial image classification method based on label-guided diversity contrastive learning, characterized by: The classification method includes the following steps: (1) Using pre-acquired low-resolution industrial image data with category labels and a diversity contrast learning strategy to train a feature extractor in a neural network model, extracting representation vectors of low-resolution images of known categories through the trained feature extractor, and using the extracted representation vectors to train a classifier in the neural network model, thereby obtaining a low-resolution image classification model; wherein the feature extractor is embedded with a sequence decoupled large kernel convolution module; (2) The low-resolution image to be classified is input into the obtained low-resolution image classification model, the feature extractor extracts features from the corresponding low-resolution image and generates a representation vector, the classifier decodes the representation vector and outputs the classification result, thereby realizing automatic classification of the low-resolution image.

2. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to claim 1, characterized in that: The feature extractor uses a convolutional neural network structure, and the classifier uses a feedforward neural network structure, which is used to decode the representation vector from the feature extractor layer by layer, thereby realizing classification of the low-resolution image.

3. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to claim 1, characterized in that: The feature extractor uses ResNet18 as the backbone, and the classifier is divided into three layers: input layer, hidden layer, and output layer.

4. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to claim 3, characterized in that: The sequence decoupling large kernel convolution module includes two parts: sequence decoupling and sequence space selection. Sequence decoupling uses depth-wise convolution sequence Explicitly decouple the large kernel convolution mechanism, then use Ghost convolution C i Ghost (·) Restore the channel suppression of each layer of depth-wise convolution process; Sequence space selection first splices the feature maps H extracted from all scale receptive fields along the channel dimension i And through parallel channels, the feature maps extracted from all scale receptive fields are spliced along the channel dimension to obtain the overall feature map, which is subjected to global average pooling and global maximum pooling. Get 2 pixel-level pooling feature matrices; Then, use the unpooling convolution The number of channels is restored to the sequence value N, and the sigmoid function is used to activate the feature map restored by the de-pooling convolution to obtain the mask weight of each large kernel decomposition sequence; finally, the de-pooling convolution is used After restoring the number of input channels, the input X connected with the residual i Make Hadamard product and get the final output value Y i .

5. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to claim 4, characterized in that: The final output value Y i The formula is: The classifier contains a dropout layer to impose regularization constraints.

6. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to any one of claims 1 to 5, characterized in that: The input to the feature extractor is the original low-resolution image x i and its corresponding category label y i , the output is a representation vector containing discriminative information The parameters θ of the neural network are optimized by minimizing the distance between the representation vectors of similar images and maximizing the distance between the representation vectors of heterogeneous images.

7. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to any one of claims 1 to 5, characterized in that: The label-guided diversity contrastive learning strategy is divided into two parts: label-guided contrastive learning and diversity contrastive learning; label-guided contrastive learning learns a low-dimensional representation by maximizing the consistency of positive sample pairs and minimizing the consistency of negative sample pairs. Assume data set There are n samples in total Then the loss function of label-guided contrastive learning is: in, and Represent the positive sample pair set and the negative sample pair set respectively, x P is a positive sample pair of sample x, x N is a negative sample pair of sample x, and f(·;θ) is the mapping relationship learned by the neural network.

8. The low-resolution industrial image classification method based on label-guided diversity contrastive learning according to claim 7, characterized in that: Diversity Contrastive Learning introduces an additional quadratic regularization term based on the Gram matrix into the loss function of label-guided contrastive learning. / 2 is used to penalize similarity and promote the orthogonalization of the weight matrix W to learn more diverse representations. The loss function of label-guided diversity contrastive learning is:

9. A low-resolution industrial image classification system based on label-guided diversity contrastive learning, characterized by: The system includes a memory and a processor, the memory stores a computer program, and the processor executes the low-resolution industrial image classification method based on label-guided diversity contrast learning according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the low-resolution industrial image classification method based on label-guided diversity contrast learning as described in any one of claims 1 to 8.

Images