A method for constructing a multi-label classification model for waste bottles and a method for classifying waste bottles

Abstract

The present invention belongs to the technical field of image processing, and in particular to a method for constructing a waste bottle multi-label classification model, comprising: obtaining a waste bottle image data set, extracting context features according to the waste bottle image data set using a residual network structure, and constructing a residual learning model; wherein the waste bottle image data set includes waste bottle image data and waste bottle image category labels; combining a channel attention mechanism with a spatial attention mechanism, extracting classification features of the waste bottle image data set, and constructing a hybrid attention model; based on a feature image output by the hybrid attention model, generating a label classification value, and constructing a self-attention model; constructing a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model, and the self-attention model; and by combining the residual learning model, the hybrid attention model, and the self-attention model, the accuracy of the waste bottle multi-label classification can be significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for constructing a multi-label classification model for waste bottles and a method for classifying waste bottles. Background Art

[0002] The existing waste bottles contain the following 8 labels: blue, green, white, other colors, broken, not broken, package removed, and package not removed. Therefore, there is no obvious dependency between the labels. For example, the probability of green and white bottles being broken should be considered the same.

[0003] To obtain the semantic relationship of multiple labels, researchers use a variety of structures to model the semantic relationship or co-occurrence between labels. For example, it is proposed to use a recurrent neural network (RNN) to model label relationships and dependencies; another example is to use a graph convolution network (Graph Convolution Network) to establish an association matrix for co-occurrence pairs of labels. Although this type of method solves the problem of exponential growth in the prediction space of training multiple independent binary classifiers, it is not very suitable for the multi-label task of garbage plastic bottles.

[0004] Secondly, researchers also use the method of generating candidate regions to select the area containing the object in the image, and then use the model to extract and classify features in the corresponding area. For example, the hypotheses-CNN-pooling (HCP) method is proposed to solve the multi-label image classification problem without the need for true value candidate boxes; in addition, the Random CropPooling (RCP) method is proposed, and its random scaling and cropping method is suitable for complex image content. Although the efficiency of generating candidate regions is significantly improved compared to the traditional sliding window method, it still takes a lot of time to generate candidate regions in actual application scenarios.

[0005] Secondly, methods based on the attention mechanism have also been proposed. For example, the Recurrent Attentional Reinforcement Learning (RARL) method uses a reinforcement learning module to iteratively generate attention regions and classification scores. Similarly, the Recurrently Discovering Attentional Regions (RDAR) uses spatial transformation to generate attention regions and then calculate classification scores. Although these two methods can save computational effort, the categories of the generated regions are unknown and still need to be improved.

[0006] Furthermore, some researchers have proposed the Spatial Regularization Network (SRN) to use attention maps to obtain the relationship and spatial semantics between labels. Although it can also reduce the amount of calculation and improve the classification performance, it is also not suitable for the waste bottle classification task due to the label relationship. In addition, the multi-label classification of waste bottles requires the model to be able to detect the damage of the bottle body. Due to the variety of damage forms, most methods that perform well in public datasets have low accuracy in the task of waste bottle classification.

[0007] Therefore, it is necessary to design a waste bottle multi-label classification model construction method and a waste bottle classification method to improve the classification efficiency and classification accuracy of waste bottle multi-label classification and reduce the amount of calculation required for waste bottle classification. Summary of the invention

[0008] The purpose of the present invention is to provide a method for constructing a multi-label classification model for waste bottles and a method for classifying waste bottles, aiming to solve the technical problems in the prior art of low classification efficiency, low classification accuracy and large amount of calculation required for waste bottle classification.

[0009] To achieve the above-mentioned purpose, an embodiment of the present invention provides a method for constructing a multi-label classification model for waste bottles, which is characterized by comprising: obtaining a waste bottle image dataset, and according to the waste bottle image dataset, extracting context features using a residual network structure to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels; combining a channel attention mechanism with a spatial attention mechanism, extracting classification features of the waste bottle image dataset, and constructing a hybrid attention model; based on a feature image output by the hybrid attention model, generating a label classification value, and constructing a self-attention model; and constructing a multi-label classification model for waste bottles according to the residual learning model, the hybrid attention model, and the self-attention model.

[0010] Optionally, the step of combining the channel attention mechanism with the spatial attention mechanism, extracting classification features of the waste bottle image dataset, and constructing a hybrid attention model specifically includes:

[0011] After constructing the residual learning module and extracting context features from the waste bottle image dataset, a channel attention unit is constructed and the feature image is input. The channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels.

[0012] A spatial attention unit is constructed and input into the feature image after the channel attention unit processes the output. The spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent.

[0013] Optionally, after constructing the residual learning module and extracting context features from the waste bottle image dataset, constructing a channel attention unit and inputting a feature image, wherein the channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels, specifically includes:

[0014] Obtaining a feature image after context features are extracted by a residual network model, inputting the feature image into the channel attention unit, and extracting channel information;

[0015] Perform maximum pooling and average pooling operations on the input feature image respectively;

[0016] Acquire feature images after the maximum pooling operation and the average pooling operation respectively, and input them into a multi-layer perceptron, wherein the multi-layer perceptron is composed of two linear connection layers and an activation function;

[0017] Perform element-wise addition operations on the feature image output by the multi-layer perceptron and convert it into a channel attention vector through the sigmoid function;

[0018] Perform element-wise multiplication of the feature image input by the residual network model and the channel attention vector to output the feature image.

[0019] Optionally, the step of constructing a spatial attention unit and inputting the feature image output after processing by the channel attention unit, wherein the spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent, specifically includes:

[0020] Obtaining a feature image output by the channel attention unit, inputting it into the spatial attention unit, and extracting spatial information;

[0021] Perform maximum pooling and average pooling operations on the input feature image respectively;

[0022] Perform channel splicing operation on the feature image after the maximum pooling operation and the average pooling operation;

[0023] The feature image after the channel splicing operation is passed into the convolution layer with a convolution kernel size of 3×3, fused into a single-channel feature image, and converted into a spatial attention vector through the sigmoid function;

[0024] The feature image input by the channel attention unit is element-wise multiplied by the spatial attention vector to output the feature image.

[0025] Optionally, the step of generating label classification values ​​based on the feature image output by the hybrid attention model and constructing a self-attention model specifically includes:

[0026] Obtain the feature image output by the hybrid attention model and input it into the self-attention model.

[0027] The input feature image is passed into the convolution layer with a convolution kernel size of 1×1, the feature image is output through convolution and the convolution layer weight parameters are obtained.

[0028] Perform a binary norm operation on the convolutional layer weight parameters to obtain the self-attention vector;

[0029] The feature image output by the convolution is element-wise divided by the self-attention vector, and the feature image after the element-wise division is averaged to obtain the classification scores of each label.

[0030] Optionally, the step of obtaining a waste bottle image dataset, extracting context features using a residual network structure according to the waste bottle image dataset, and constructing a residual learning model specifically includes:

[0031] Obtain a waste bottle image dataset, and input the feature map of the waste bottle image dataset into the weight layer of the residual network structure, where the weight layer of the residual network structure consists of multiple convolutional layers, BN layers, and ReLU layers;

[0032] The feature image output by the weight layer of the residual network structure is sent to the maximum pooling layer for maximum pooling operation;

[0033] The feature image output by the maximum pooling layer is output to multiple Res layers in sequence, and the multiple Res layers are composed of bottleneck layers to extract the features of the feature image.

[0034] Optionally, the residual network structure is ResNet101.

[0035] The present invention also provides a waste bottle classification method, comprising:

[0036] Acquire a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0037] Loading the waste bottle multi-label classification model constructed by the waste bottle multi-label classification model construction method, inputting the waste bottle image dataset into the waste bottle multi-label classification model in batches, and training the waste bottle model;

[0038] The image to be classified is input into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

[0039] The present invention also provides a waste bottle multi-label classification model construction device, comprising:

[0040] A residual learning model construction module, wherein the residual learning model construction module is used to obtain a waste bottle image dataset, and according to the waste bottle image dataset, extract context features using a residual network structure to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0041] A hybrid attention model building module, wherein the hybrid attention model building module is used to combine the channel attention mechanism and the spatial attention mechanism, extract classification features of the waste bottle image dataset, and build a hybrid attention model;

[0042] A self-attention model construction module, wherein the self-attention model construction module is used to generate a label classification value based on the feature image output by the hybrid attention model and construct a self-attention model;

[0043] A waste bottle multi-label classification model construction module is used to construct a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model and the self-attention model.

[0044] The present invention also provides a waste bottle classification device, comprising:

[0045] An image feature acquisition module, wherein the image feature acquisition module is used to acquire a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0046] A waste bottle multi-label classification model training module, wherein the waste bottle multi-label classification model training module is used to load the waste bottle multi-label classification model constructed by the waste bottle multi-label classification model construction method, input the waste bottle image data set into the waste bottle multi-label classification model in batches, and train the waste bottle model;

[0047] The image recognition module is used to input the image to be classified into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

[0048] The above one or more technical solutions in the method for building a multi-label classification model for waste bottles provided by the embodiment of the present invention have at least one of the following technical effects:

[0049] The present invention sets a residual learning model, extracts context features of waste bottles through a pre-trained residual network structure, enables the residual learning model to learn prior knowledge, shortens the training time required for the waste bottle multi-label classification model, improves the convergence effect of the waste bottle model, reduces the size of the training data set required for training, and greatly reduces the cost of manually annotating data sets; sets a hybrid attention model combining a channel attention mechanism and a spatial attention mechanism, extracts more significant classification features from the feature image; and sets the self-attention model to obtain the label classification value. The self-attention mechanism uses its own parameters to calculate the attention vector, combines the feature map to improve the classification effect, and does not need to add additional parameters, thereby reducing the number of model parameters. By comparing the label classification value with a preset standard threshold and converting it into a corresponding label, the category corresponding to the waste bottle is obtained. Therefore, the waste bottle multi-label classification model can significantly improve the accuracy of waste bottle multi-label classification by combining the residual learning model, the hybrid attention model and the self-attention model.

[0050] The above one or more technical solutions in the waste bottle classification method provided by the embodiment of the present invention have at least one of the following technical effects:

[0051] By inputting the waste bottle image into the waste bottle multi-label classification model, the waste bottle multi-label classification model is trained, so that the classified waste bottle multi-label classification model can recognize the image to be classified, obtain the recognition result of the image to be classified, and effectively improve the accuracy of the waste bottle multi-label classification. Because the waste bottle classification method applies the waste bottle multi-label classification model, the waste bottle classification method also has the technical effect of the waste bottle multi-label classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flowchart of a method for constructing a multi-label classification model for waste bottles provided in an embodiment of the present invention;

[0054] Figure 2 A flowchart of constructing a residual learning model provided by an embodiment of the present invention;

[0055] Figure 3 A flowchart of constructing a channel attention unit and a spatial attention unit provided by an embodiment of the present invention;

[0056] Figure 4A flow chart of constructing a channel attention unit provided by an embodiment of the present invention;

[0057] Figure 5 A flowchart of constructing a spatial attention unit provided by an embodiment of the present invention;

[0058] Figure 6 A flowchart of constructing a self-attention model provided by an embodiment of the present invention;

[0059] Figure 7 A flowchart of a waste bottle classification method provided by an embodiment of the present invention;

[0060] Figure 8 A structural block diagram of a device for constructing a multi-label classification model for waste bottles provided in an embodiment of the present invention;

[0061] Fig. 9 A structural block diagram of a waste bottle classification device provided by an embodiment of the present invention;

[0062] Fig.10 A structural block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0064] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0065] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0066] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0067] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0068] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0069] In one embodiment of the present invention, Figure 1 As shown, a method for constructing a multi-label classification model for waste bottles is provided, which is characterized by comprising:

[0070] S100: Obtain a waste bottle image dataset, and extract context features using a residual network structure according to the waste bottle image dataset to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0071] S200: Combine the channel attention mechanism with the spatial attention mechanism to extract the classification features of the waste bottle image dataset and build a hybrid attention model;

[0072] S300: Generate label classification values ​​based on the feature image output by the hybrid attention model and build a self-attention model;

[0073] S400: Constructing a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model and the self-attention model.

[0074] The present invention sets a residual learning model, extracts context features of waste bottles through a pre-trained residual network structure, enables the residual learning model to learn prior knowledge, shortens the training time required for the waste bottle multi-label classification model, improves the convergence effect of the waste bottle model, reduces the size of the training data set required for training, and greatly reduces the cost of manually annotating data sets; sets a hybrid attention model combining a channel attention mechanism and a spatial attention mechanism, extracts more significant classification features from the feature image; and sets the self-attention model to obtain the label classification value. The self-attention mechanism uses its own parameters to calculate the attention vector, combines the feature map to improve the classification effect, and does not need to add additional parameters, thereby reducing the number of model parameters. By comparing the label classification value with a preset standard threshold and converting it into a corresponding label, the category corresponding to the waste bottle is obtained. Therefore, the waste bottle multi-label classification model can significantly improve the accuracy of waste bottle multi-label classification by combining the residual learning model, the hybrid attention model and the self-attention model.

[0075] Specifically, in another embodiment of the present invention, Figure 3 As shown, the steps of combining the channel attention mechanism and the spatial attention mechanism to extract the classification features of the waste bottle image dataset and construct a hybrid attention model include:

[0076] S210: after constructing the residual learning module and extracting context features from the waste bottle image dataset, constructing a channel attention unit and inputting a feature image, wherein the channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels;

[0077] S220: Construct a spatial attention unit and input the feature image after processing the output of the channel attention unit. The spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent.

[0078] The channel attention unit is responsible for balancing the weights of different channels in the output feature map, and the channel attention unit can help the hybrid attention model learn the types of waste bottles. The spatial attention unit enables the hybrid attention model to focus on important spatial positions, so that the spatial attention unit can help the hybrid attention model learn the key positions of waste bottles. Therefore, it is necessary to combine the channel attention unit and the spatial attention unit into a hybrid attention model.

[0079] Specifically, in another embodiment of the present invention, Figure 4As shown, after constructing the residual learning module and extracting context features from the waste bottle image dataset, constructing a channel attention unit and inputting a feature image, the channel attention unit is used to weight the feature image, and the step of balancing the weights of the feature images output by the residual learning model in different channels specifically includes:

[0080] S211: Obtain a feature image after context features are extracted by a residual network model, input it into the channel attention unit, and extract channel information;

[0081] S212: performing maximum pooling and average pooling operations on the input feature image respectively;

[0082] Specifically, in this step, the input feature image is subjected to the maximum pooling and average pooling operations in parallel, and the channel information of the feature image is aggregated by using average pooling and maximum pooling. Two output results are generated, representing the average pooling feature and the maximum pooling feature, respectively, and then the two output features are passed to the multi-layer perceptron in parallel.

[0083] S213: acquiring feature images after the maximum pooling operation and the average pooling operation respectively, and inputting them into a multi-layer perceptron, wherein the multi-layer perceptron is composed of two linear connection layers and an activation function;

[0084] In this step, the linear connection layer includes a linear input connection layer and a linear output connection layer, the linear input connection layer receives the output result of the pooling operation and outputs it to the activation function ReLu, the activation function ReLu is output to the linear output connection layer, and the linear output connection layer outputs the feature vector. The number of output channels of the linear input connection layer and the linear output connection layer are both 2048.

[0085] S214: perform element-wise addition operation on the feature image output by the multi-layer perceptron, and convert it into a channel attention vector through a sigmoid function;

[0086] In this step, the feature vectors output from the linear output connection layer are merged by using the element-wise summation method to generate a feature image of the channel attention, and the channel attention vector is obtained by combining the feature image of the channel attention with the sigmoid function calculation.

[0087] S215: Perform element-wise multiplication of the feature image input by the residual network model and the channel attention vector to output the feature image.

[0088] In this step, the feature image when entering the channel attention unit is obtained, that is, the feature image input from the residual network model, and the feature image is element-wise multiplied with the channel attention vector to obtain the input feature image required by the spatial attention unit.

[0089] Specifically, in another embodiment of the present invention, Figure 5 As shown, the step of constructing a spatial attention unit and inputting the feature image after processing the output of the channel attention unit, wherein the spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent, specifically includes:

[0090] S221: Obtain a feature image output by the channel attention unit, input it into the spatial attention unit, and extract spatial information;

[0091] S222: performing maximum pooling and average pooling operations on the input feature image respectively;

[0092] S223: performing a channel splicing operation on the feature image after the maximum pooling operation and the average pooling operation;

[0093] S224: passing the feature image after the channel splicing operation into a convolution layer with a convolution kernel size of 3×3, fusing it into a single-channel feature image, and converting the feature image into a spatial attention vector through a sigmoid function;

[0094] S225: Perform element-wise multiplication of the feature image input by the channel attention unit and the spatial attention vector, and output the feature image.

[0095] Specifically, the spatial attention map is generated by utilizing the spatial relationship between feature images, performing average pooling and maximum pooling operations, generating average pooling features and maximum pooling features on the channel, and performing channel splicing operations on the average pooling features and the maximum pooling features, which are fused into a single channel feature map by a convolution layer, wherein the convolution kernel size of the convolution layer is 3×3, the number of input channels is 2, and the number of output channels is 1. Similarly, by using the sigmoid function, the feature image is converted into a spatial attention vector, and then the spatial attention vector is element-wise multiplied with the feature image of the channel attention unit input to the spatial attention unit to generate the feature image required by the self-attention model.

[0096] Specifically, in another embodiment of the present invention, Figure 6 As shown, the step of generating label classification values ​​based on the feature image output by the hybrid attention model and constructing the self-attention model specifically includes:

[0097] S310: Obtain the feature image output by the hybrid attention model and input it into the self-attention model.

[0098] S320: The input feature image is passed into a convolution layer with a convolution kernel size of 1×1, the feature image is output through convolution and the weight parameters of the convolution layer are obtained, where the number of input channels is 2048, the number of output channels is c, and c is the number of categories.

[0099] S330: Perform a binary norm operation on the weight parameters of the convolution layer to obtain a self-attention vector;

[0100] S340: Perform an element-wise division operation on the feature image output by the convolution and the self-attention vector, and perform an average operation on the feature image after the element-wise division operation to obtain the classification score of each label.

[0101] The self-attention model generates a label classification score based on the feature map output by the hybrid attention model. The self-attention mechanism uses its own parameters to calculate the attention vector, and combines the feature map to improve the classification effect without adding additional parameters, thereby reducing the number of parameters of the multi-label classification model for waste bottles. The self-attention model compares the label classification score with a preset standard threshold and converts it into a corresponding label. Thus, the self-attention model improves the accuracy of generating each label classification score without increasing the number of parameters of the multi-label classification model for waste bottles. Specifically, in one embodiment, the preset standard threshold for the damaged label is that the label classification score is greater than 0.5. When the generated label classification score is greater than 0.5, it is considered that the image belongs to the category of the damaged label.

[0102] Specifically, in another embodiment of the present invention, Figure 2 As shown, the steps of obtaining a waste bottle image dataset, extracting context features using a residual network structure according to the waste bottle image dataset, and constructing a residual learning model specifically include:

[0103] S110: Obtain a waste bottle image dataset, and input a feature map of the waste bottle image dataset into a weight layer of a residual network structure, wherein the weight layer of the residual network structure is composed of a plurality of convolutional layers, a BN layer, and a ReLU layer; further, the residual network structure is ResNet101.

[0104] S120: Sending the feature image output by the weight layer of the residual network structure to the maximum pooling layer for performing a maximum pooling operation;

[0105] S130: Output the feature image output by the maximum pooling layer to multiple Res layers in sequence, wherein the multiple Res layers are composed of bottleneck layers, and extract the features of the feature image. In this step, the number of Res layers is 4, the first layer of the Res layer includes 3 bottleneck layers, the second layer of the Res layer includes 4 bottleneck layers, the third layer of the Res layer includes 23 bottleneck layers, and the fourth layer of the Res layer includes 3 bottleneck layers. By setting a deeper network structure, the complex features in the waste bottle image data set can be better extracted, and by selecting a residual network structure, the phenomenon of degradation when the number of network layers is too deep can be effectively prevented.

[0106] Specifically, by adopting the pre-trained residual network, the size of the data set required for model training is greatly reduced, which effectively saves the time required for training, makes the multi-label classification model of waste bottles converge better, and reduces the manual workload of annotating the data set.

[0107] The present invention also provides a waste bottle classification method, comprising:

[0108] S500: Obtain a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels. In one embodiment, the waste bottle image dataset includes 3224 images, each of which is binary labeled according to the following eight labels: blue, green, white, other colors, damaged, undamaged, package removed, and package not removed.

[0109] S600: Load the waste bottle multi-label classification model constructed by the waste bottle multi-label classification model construction method, input the waste bottle image dataset into the waste bottle multi-label classification model in batches, and train the waste bottle model. Specifically, in this step, after the waste bottle multi-label classification model is constructed, the waste bottle image dataset is input into the waste bottle multi-label classification model in batches. The waste bottle image dataset is input into the waste bottle multi-label classification model, and the loss is calculated by the binary cross entropy loss function (BCELoss) and iterated repeatedly to optimize the waste bottle multi-label classification model.

[0110] S700: Input the image to be classified into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

[0111] By inputting the waste bottle image into the waste bottle multi-label classification model, the waste bottle multi-label classification model is trained, so that the classified waste bottle multi-label classification model can recognize the image to be classified, obtain the recognition result of the image to be classified, and effectively improve the accuracy of the waste bottle multi-label classification. Because the waste bottle classification method applies the waste bottle multi-label classification model, the waste bottle classification method also has the technical effect of the waste bottle multi-label classification model.

[0112] The present invention also provides a device for constructing an attention sequence framework model for waste bottle classification, such as Figure 8 As shown, including:

[0113] A residual learning model construction module, wherein the residual learning model construction module is used to obtain a waste bottle image dataset, and according to the waste bottle image dataset, extract context features using a residual network structure to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0114] A hybrid attention model building module, wherein the hybrid attention model building module is used to combine the channel attention mechanism and the spatial attention mechanism, extract classification features of the waste bottle image dataset, and build a hybrid attention model;

[0115] A self-attention model construction module, wherein the self-attention model construction module is used to generate a label classification value based on the feature image output by the hybrid attention model and construct a self-attention model;

[0116] A waste bottle multi-label classification model construction module is used to construct a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model and the self-attention model.

[0117] In another embodiment of the present invention, the hybrid attention model building module is further used to:

[0118] After constructing the residual learning module and extracting context features from the waste bottle image dataset, a channel attention unit is constructed and the feature image is input. The channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels.

[0119] A spatial attention unit is constructed and input into the feature image after the channel attention unit processes the output. The spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent.

[0120] In another embodiment of the present invention, the hybrid attention model building module is further used to:

[0121] Obtaining a feature image after context features are extracted by a residual network model, inputting the feature image into the channel attention unit, and extracting channel information;

[0122] Perform maximum pooling and average pooling operations on the input feature image respectively;

[0123] Acquire feature images after the maximum pooling operation and the average pooling operation respectively, and input them into a multi-layer perceptron, wherein the multi-layer perceptron is composed of two linear connection layers and an activation function;

[0124] Perform element-wise addition operations on the feature image output by the multi-layer perceptron and convert it into a channel attention vector through the sigmoid function;

[0125] Perform element-wise multiplication of the feature image input by the residual network model and the channel attention vector to output the feature image.

[0126] In another embodiment of the present invention, the hybrid attention model building module is further used to:

[0127] Obtaining a feature image output by the channel attention unit, inputting it into the spatial attention unit, and extracting spatial information;

[0128] Perform maximum pooling and average pooling operations on the input feature image respectively;

[0129] Perform channel splicing operation on the feature image after the maximum pooling operation and the average pooling operation;

[0130] The feature image after the channel splicing operation is passed into the convolution layer with a convolution kernel size of 3×3, fused into a single-channel feature image, and converted into a spatial attention vector through the sigmoid function;

[0131] The feature image input by the channel attention unit is element-wise multiplied by the spatial attention vector to output the feature image.

[0132] In another embodiment of the present invention, the self-attention model building module is further used to:

[0133] Obtain the feature image output by the hybrid attention model and input it into the self-attention model.

[0134] The input feature image is passed into the convolution layer with a convolution kernel size of 1×1, and the feature image is output through convolution and the convolution layer weight parameters are obtained. The number of input channels is 2048, the number of output channels is c, and c is the number of categories.

[0135] Perform a binary norm operation on the convolutional layer weight parameters to obtain the self-attention vector;

[0136] The feature image output by the convolution is element-wise divided by the self-attention vector, and the feature image after the element-wise division is averaged to obtain the classification scores of each label.

[0137] In another embodiment of the present invention, the residual learning model construction module is further used for:

[0138] Obtain a waste bottle image dataset, and input the feature map of the waste bottle image dataset into the weight layer of the residual network structure, where the weight layer of the residual network structure consists of multiple convolutional layers, BN layers, and ReLU layers;

[0139] The feature image output by the weight layer of the residual network structure is sent to the maximum pooling layer for maximum pooling operation;

[0140] The feature image output by the maximum pooling layer is output to multiple Res layers in sequence, and the multiple Res layers are composed of bottleneck layers to extract the features of the feature image.

[0141] The present invention also provides a waste bottle sorting device, such as Fig. 9 As shown, including:

[0142] An image feature acquisition module, wherein the image feature acquisition module is used to acquire a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels;

[0143] A waste bottle multi-label classification model training module, wherein the waste bottle multi-label classification model training module is used to load the waste bottle multi-label classification model constructed by the waste bottle multi-label classification model construction method, input the waste bottle image data set into the waste bottle multi-label classification model in batches, and train the waste bottle model;

[0144] The image recognition module is used to input the image to be classified into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

[0145] In another embodiment of the present invention, a computer device is also provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the programs include control instructions for executing a method for building a multi-label classification model for waste bottles.

[0146] The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for constructing a multi-label classification model for waste bottles is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0147] In another embodiment of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for constructing a multi-label classification model for waste bottles are implemented.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0149] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a multi-label classification model for waste bottles, characterized in that: include: S100: Obtain a waste bottle image dataset, and extract context features using a residual network structure according to the waste bottle image dataset to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels; S200: Combine the channel attention mechanism with the spatial attention mechanism to extract the classification features of the waste bottle image dataset and build a hybrid attention model; S300: Generate label classification values ​​based on the feature image output by the hybrid attention model and build a self-attention model; S400: Constructing a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model and the self-attention model.

2. The method for constructing a multi-label classification model for waste bottles according to claim 1, characterized in that: The step of combining the channel attention mechanism and the spatial attention mechanism to extract the classification features of the waste bottle image dataset and construct a hybrid attention model includes: S210: after constructing the residual learning module and extracting context features from the waste bottle image dataset, constructing a channel attention unit and inputting a feature image, wherein the channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels; S220: Construct a spatial attention unit and input the feature image after processing the output of the channel attention unit. The spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent.

3. The method for constructing a multi-label classification model for waste bottles according to claim 2, characterized in that: After constructing the residual learning module and extracting context features from the waste bottle image dataset, constructing a channel attention unit and inputting a feature image, wherein the channel attention unit is used to weight the feature image and balance the weights of the feature images output by the residual learning model in different channels, specifically includes: S211: Obtain a feature image after context features are extracted by a residual network model, input it into the channel attention unit, and extract channel information; S212: performing maximum pooling and average pooling operations on the input feature image respectively; S213: acquiring feature images after the maximum pooling operation and the average pooling operation respectively, and inputting them into a multi-layer perceptron, wherein the multi-layer perceptron is composed of two linear connection layers and an activation function; S214: perform element-wise addition operation on the feature image output by the multi-layer perceptron, and convert it into a channel attention vector through a sigmoid function; S215: Perform element-wise multiplication of the feature image input by the residual network model and the channel attention vector to output the feature image.

4. The method for constructing a multi-label classification model for waste bottles according to claim 3, characterized in that: The step of constructing a spatial attention unit and inputting the feature image processed and output by the channel attention unit, wherein the spatial attention unit is used to generate corresponding weights for the positions of the feature images, extract key spatial positions in the feature images, and weight the feature images to make the classification features of the feature images more prominent, specifically includes: S221: Obtain a feature image output by the channel attention unit, input it into the spatial attention unit, and extract spatial information; S222: performing maximum pooling and average pooling operations on the input feature image respectively; S223: performing a channel splicing operation on the feature image after the maximum pooling operation and the average pooling operation; S224: passing the feature image after the channel splicing operation into a convolution layer with a convolution kernel size of 3×3, fusing it into a single-channel feature image, and converting the feature image into a spatial attention vector through a sigmoid function; S225: Perform element-wise multiplication of the feature image input by the channel attention unit and the spatial attention vector, and output the feature image.

5. The method for constructing a multi-label classification model for waste bottles according to claim 1, characterized in that: The step of generating label classification values ​​based on the feature image output by the hybrid attention model and constructing a self-attention model specifically includes: S310: Obtain the feature image output by the hybrid attention model and input it into the self-attention model. S320: The input feature image is passed into a convolution layer with a convolution kernel size of 1×1, the feature image is output through convolution and the weight parameters of the convolution layer are obtained. S330: Perform a binary norm operation on the weight parameters of the convolution layer to obtain a self-attention vector; S340: Perform an element-wise division operation on the feature image output by the convolution and the self-attention vector, and perform an average operation on the feature image after the element-wise division operation to obtain the classification score of each label.

6. The method for constructing a multi-label classification model for waste bottles according to claim 1, characterized in that: The step of obtaining a waste bottle image dataset, extracting context features using a residual network structure according to the waste bottle image dataset, and constructing a residual learning model specifically includes: S110: Obtain a waste bottle image dataset, and input a feature map of the waste bottle image dataset into a weight layer of a residual network structure, where the weight layer of the residual network structure is composed of multiple convolutional layers, BN layers, and ReLU layers; S120: Sending the feature image output by the weight layer of the residual network structure to the maximum pooling layer for performing a maximum pooling operation; S130: Outputting the feature image output by the maximum pooling layer to a plurality of Res layers in sequence, wherein the plurality of Res layers are composed of a bottleneck layer, and extracting features of the feature image.

7. The method for constructing a multi-label classification model for waste bottles according to claim 6, characterized in that: The residual network structure is ResNet101.

8. A method for sorting waste bottles, characterized in that: include: S500: Acquire a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels; S600: loading the waste bottle multi-label classification model constructed by the method for constructing a waste bottle multi-label classification model according to any one of claims 1 to 7, inputting the waste bottle image dataset into the waste bottle multi-label classification model in batches, and training the waste bottle model; S700: Input the image to be classified into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

9. A device for constructing an attention sequence framework model for waste bottle classification, characterized in that: include: A residual learning model construction module, wherein the residual learning model construction module is used to obtain a waste bottle image dataset, and according to the waste bottle image dataset, extract context features using a residual network structure to construct a residual learning model; wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels; A hybrid attention model building module, wherein the hybrid attention model building module is used to combine the channel attention mechanism and the spatial attention mechanism, extract classification features of the waste bottle image dataset, and build a hybrid attention model; A self-attention model construction module, wherein the self-attention model construction module is used to generate a label classification value based on the feature image output by the hybrid attention model and construct a self-attention model; A waste bottle multi-label classification model construction module is used to construct a waste bottle multi-label classification model according to the residual learning model, the hybrid attention model and the self-attention model.

10. A waste bottle sorting device, characterized in that: include: An image feature acquisition module, wherein the image feature acquisition module is used to acquire a waste bottle image dataset, wherein the waste bottle image dataset includes waste bottle image data and waste bottle image category labels; A waste bottle multi-label classification model training module, wherein the waste bottle multi-label classification model training module is used to load the waste bottle multi-label classification model constructed by the waste bottle multi-label classification model construction method according to any one of claims 1 to 7, input the waste bottle image data set into the waste bottle multi-label classification model in batches, and train the waste bottle model; The image recognition module is used to input the image to be classified into the trained waste bottle multi-label classification model to obtain the recognition result of the image to be classified.

Images