Garbage classification method based on weak sample learning

Abstract

The application relates to a garbage classification method based on weak sample learning. In a model training stage, data is divided into a few strongly-labeled basic category data and a plurality of weakly-labeled new category data, and model training is completed through two training stages of a similarity network and a main classifier, so that the quality requirement of data labeling of the model is reduced, the existing data is more easily used to complete model training, and data labeling work is simplified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a garbage classification method based on weak sample learning. BACKGROUND

[0002] The past four decades have seen rapid development of productivity, which has enabled mass production of various items and their entry into millions of households. With the rapid development of China's social economy and the continuous advancement of urbanization, the population is constantly gathering in large cities, and various materials are gathering in cities at the same time. The past garbage disposal methods of incineration and landfill have long been inadequate in the face of population concentration and rapid development of productivity, and these methods also have serious environmental pollution, land occupation, resource waste and other problems.

[0003] Under such a background, the concept of garbage classification emerged. For residents, the biggest problem in the implementation of garbage classification is that they do not know how to classify garbage. The government and many enterprises have successively introduced many methods to popularize relevant knowledge, which has achieved some results, but the problem still exists. Today, one way is to combine image processing technology to achieve it through intelligent garbage cans. The garbage picture is obtained by the camera, and the type of garbage is identified by the image processing algorithm. ResNet50, VGG16, Transformer and other algorithms have been used in this scenario. The important problem existing in the algorithm model training is the data problem and the deployment problem. As for the data problem, on the one hand, garbage classification is a relatively subdivided and single application scenario, and there are few public data sets with high labeling quality; on the other hand, if all training data is manually labeled, it is time-consuming and inefficient. SUMMARY

[0004] In order to overcome at least one of the deficiencies in the prior art, the present application provides a garbage classification method based on weak sample learning.

[0005] In a first aspect, a garbage classification model construction method based on weak sample learning is provided, comprising:

[0006] Strongly labeling a plurality of pictures belonging to the kitchen waste type to obtain a basic category data set;

[0007] Weakly labeling a plurality of pictures belonging to the non-kitchen waste type to obtain a new category data set; the non-kitchen waste type includes recyclable waste type, hazardous waste type and other waste type;

[0008] Extracting picture features of pictures in the basic category data set and picture features of pictures in the new category data set based on a backbone network;

[0009] The picture features of the pictures in the base category dataset and the picture features of the pictures in the new category dataset are input into the similarity network, the similarity network is trained based on the discriminator, and a trained similarity network is obtained;

[0010] For each non-kitchen garbage type, the similarity between any two pictures belonging to the non-kitchen garbage type is calculated based on the trained similarity network, and a similarity matrix corresponding to the non-kitchen garbage type is obtained; and the similarity matrix is used to train a main classifier, and a trained main classifier is obtained.

[0011] The backbone network, the trained similarity network, and the trained main classifier constitute a garbage classification model based on weak sample learning.

[0012] In one embodiment, the similarity network is trained based on the discriminator, including:

[0013] First, the similarity network is frozen to minimize the adversarial loss of the discriminator, and the calculation formula of the adversarial loss is:

[0014]

[0015] Wherein, L D is the adversarial loss, M is the batch size of the input pictures, is the discriminator score when the relationship feature is , is the relationship feature of the image pair composed of the i-th picture and the j-th picture in the base category dataset b, is the discriminator score when the relationship feature is , is the relationship feature of the image pair composed of the i-th picture and the j-th picture in the new category dataset n; Then, the discriminator is frozen to minimize the relationship classification loss of the similarity network and maximize the adversarial loss of the discriminator, and the calculation formula of the classification loss is:

[0016]

[0017]

[0018] Wherein, L G is the classification loss, β is the weighting parameter, c i,j is the relationship category label, is the similarity score between the i-th picture and the j-th picture in the base category dataset b.

[0019] In one embodiment, the similarity matrix is used to train the main classifier, and the trained main classifier, including:

[0020] The cost function weight and the regularization loss are calculated using the similarity matrix; ​​

[0021] According to the cost function weight, a weighted classification loss is calculated;

[0022] According to the regularization loss and the weighted classification loss, a final classification loss is determined, and a calculation formula is as follows:

[0023] L 最终 = L cls_ω + αL reg

[0024] Wherein, L 最终 is the final classification loss, L cls_ω is the weighted classification loss, L reg is the regularization loss, and α is a hyperparameter;

[0025] When the final classification loss is the minimum, the trained main classifier is obtained.

[0026] In an embodiment, the cost function weight is calculated by using the following formula:

[0027]

[0028] Wherein, ω c,i represents the cost function weight of the i-th picture, is the number of pictures belonging to the non-kitchen garbage type currently processed, S c,i,j represents the similarity score of the i-th picture and the j-th picture, S c,j,i represents the similarity score of the j-th picture and the i-th picture;

[0029] The regularization loss is calculated by using the following formula:

[0030]

[0031] Wherein, L reg is the regularization loss, s i,j is each element in the similarity matrix S c , h(x i ) is the picture feature of the i-th picture x i , and h(x j ) is the picture feature of the j-th picture x j .

[0032] In an embodiment, the weighted classification loss is calculated by using the following formula:

[0033]

[0034] Wherein, L cls_ω is the weighted classification loss, is the number of pictures belonging to the non-kitchen garbage type currently processed, and is the normalized cost function weight of the ith picture, x i is the ith picture, y(x i ) represents the classification score of the picture x i belongs to its corresponding garbage type.

[0035] In a second aspect, a garbage classification method based on weak samples is provided, comprising:

[0036] inputting a garbage picture to be identified into a garbage classification model based on weak samples to obtain a classification score of the garbage picture to be identified belonging to each garbage type;

[0037] determining the garbage type of the garbage picture to be identified according to the classification score;

[0038] The garbage classification model based on weak samples is obtained according to any one of claims 1-5.

[0039] In a third aspect, a garbage classification model construction device based on weak sample learning is provided, comprising:

[0040] A basic category dataset acquisition module is configured to perform strong labeling on a plurality of pictures belonging to the kitchen waste type to obtain a basic category dataset.

[0041] A new category dataset acquisition module is configured to perform weak labeling on a plurality of pictures belonging to a non-kitchen waste type to obtain a new category dataset; the non-kitchen waste type includes a recyclable waste type, a hazardous waste type, and other waste types.

[0042] A picture feature extraction module is configured to extract picture features of pictures in the basic category dataset and picture features of pictures in the new category dataset based on a backbone network.

[0043] A similarity network training module is configured to input the picture features of the pictures in the basic category dataset and the picture features of the pictures in the new category dataset into a similarity network, train the similarity network based on a discriminator, and obtain a trained similarity network.

[0044] A main classifier training module is configured to, for each non-kitchen waste type, calculate a similarity between any two pictures belonging to the non-kitchen waste type based on the trained similarity network, obtain a similarity matrix corresponding to the non-kitchen waste type, and train a main classifier using the similarity matrix to obtain a trained main classifier.

[0045] A model construction module is configured to construct a garbage classification model based on weak sample learning using the backbone network, the trained similarity network, and the trained main classifier.

[0046] In an embodiment, the similarity network training module is further configured to:

[0047] First, freeze the similarity network to minimize the adversarial loss of the discriminator. The formula for calculating the adversarial loss is:

[0048]

[0049] Among them, L D To combat the loss, M is the batch size of the input images. The relation features are Discriminator score at time The relational features of the image pairs consisting of the i-th and j-th images in the basic dataset b are as follows: The relation features are Discriminator score at time The relational features of the image pair consisting of the i-th image and the j-th image in the new type dataset n;

[0050] Then, the discriminator is frozen to minimize the relationship classification loss of the similarity network and maximize the adversarial loss of the discriminator. The formula for calculating the classification loss is:

[0051]

[0052] Among them, L G For classification loss, β is the trade-off parameter, and c i,j For relationship category labels, It is the similarity score between the i-th image and the j-th image in the basic category dataset b.

[0053] In one embodiment, the main classifier training module is further configured to:

[0054] The cost function weights and regularization loss are calculated using a similarity matrix.

[0055] Calculate the weighted classification loss based on the cost function weights;

[0056] The final classification loss is determined based on the regularization loss and the weighted classification loss, using the following formula:

[0057] L 最终 =L cls_ω +αL reg

[0058] Among them, L 最终 For the final classification loss, L cls_ω For weighted classification loss, L reg The loss is the regularization loss, and α is a hyperparameter.

[0059] The trained main classifier is obtained when the final classification loss is minimized.

[0060] In an embodiment, the main classifier training module is further configured to:

[0061] The cost function weight is calculated by the following formula:

[0062]

[0063] wherein ω c,i represents the cost function weight of the i-th picture, is the number of pictures belonging to the non-kitchen waste type currently processed, S c,i,j represents the similarity score of the i-th picture and the j-th picture, S c,j,i represents the similarity score of the j-th picture and the i-th picture;

[0064] The regularization loss is calculated by the following formula:

[0065]

[0066] wherein L reg is the regularization loss, s i,j is each element in the similarity matrix S c , h(x i ) is the picture feature of the i-th picture x i , and h(x j ) is the picture feature of the j-th picture x j .

[0067] Compared with the prior art, the present application has the following beneficial effects: the present application significantly reduces the demand for high-quality labeled data in the model training stage by dividing the data into two categories of strong labeled data and weak labeled data; only the type of the weak labeled data needs to be labeled, and a large amount of effective data can be quickly obtained by keyword crawling of Internet pictures, and the actual model training only needs to label the strong labeled data, which can significantly reduce the data labeling cost. BRIEF DESCRIPTION OF DRAWINGS

[0068] The present application can be better understood by referring to the following description in conjunction with the accompanying drawings, which form a part of this specification and are included to further illustrate and explain the principles of the application. In the drawings:

[0069] Figure 1 Fig. 1 shows a garbage classification model construction method based on weak sample learning according to an embodiment of the present application;

[0070] Figure 2 Fig. 2 shows a structure block diagram of a garbage classification model construction device based on weak samples according to an embodiment of the present application. DETAILED DESCRIPTION

[0071] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. For the purpose of clarity and a concise description, all the features of the practical embodiments are not described in the specification. It should be appreciated, however, that many embodiment-specific decisions can be made in the process of developing any such practical embodiments in order to achieve the specific objectives of the developers, and these decisions can vary from embodiment to embodiment.

[0072] It should also be noted herein that, in order to avoid obscuring the present application with unnecessary details, only the structures of the devices closely related to the scheme according to the present application are shown in the accompanying drawings, and other details not closely related to the present application are omitted.

[0073] It should be understood that the present application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this context, the embodiments can be combined with each other, features can be replaced or borrowed between different embodiments, and one or more features can be omitted in one embodiment.

[0074] The present application provides a garbage classification method based on weak sample learning. In a model training stage, data is divided into a small number of strongly labeled basic category data and a large number of weakly labeled new category data. Model training is completed through two training stages of a similarity network and a main classifier, which reduces the quality requirement of data labeling for the model, and makes it easier to use existing data to complete model training and simplify data labeling work.

[0075] The present application provides a garbage classification model construction method based on weak sample learning, Figure 1 A garbage classification model construction method based on weak sample learning according to an embodiment of the present application is shown. The method comprises:

[0076] Step S1: a plurality of pictures belonging to the kitchen waste type are strongly labeled to obtain a basic category data set.

[0077] In this step, a plurality of pictures belonging to the kitchen waste type are selected, and each picture is strongly labeled. The strong labeling can specifically be labeling the category of the object contained in the picture. The labeled picture forms a basic category data set.

[0078] Step S2: a plurality of pictures belonging to the non-kitchen waste type are weakly labeled to obtain a new category data set. The non-kitchen waste type includes the recyclable waste type, the hazardous waste type, and the other waste type.

[0079] In this step, the keywords of "recyclable", "hazardous waste" and "other waste" can be used for search, and the related pictures can be crawled by the crawler. The crawled pictures are labeled as weak labels, i.e., the pictures are labeled as recyclable waste, hazardous waste or other waste. The labeled pictures form a new class dataset. It should be noted that the pictures in the new class dataset are not repeated with the pictures in the basic class dataset.

[0080] In step S3, the picture features of the pictures in the basic class dataset and the picture features of the pictures in the new class dataset are extracted based on the backbone network.

[0081] In this step, the picture features of the pictures in the basic class dataset and the picture features of the pictures in the new class dataset are extracted based on the backbone network.

[0082] In step S4, the picture features of the pictures in the basic class dataset and the picture features of the pictures in the new class dataset are input into the similarity network, and the similarity network is trained based on the discriminator to obtain the trained similarity network.

[0083] In step S5, for each non-kitchen waste type, the similarity between any two pictures belonging to the non-kitchen waste type is calculated based on the trained similarity network to obtain a similarity matrix corresponding to the non-kitchen waste type. The similarity matrix is used to train the main classifier to obtain the trained main classifier.

[0084] In this step, for each non-kitchen waste type, the similarity matrix is calculated to train the main classifier. After the training of the main classifier by all non-kitchen waste types is completed, the trained main classifier is obtained.

[0085] In step S6, the backbone network, the trained similarity network and the trained main classifier constitute a garbage classification model based on weak sample learning.

[0086] In this embodiment, by dividing the data into strong labeled data and weak labeled data, the demand for high-quality labeled data in the model training stage is significantly reduced. For weakly labeled data, only the type needs to be labeled, and a large amount of effective data can be quickly obtained by crawling Internet pictures through keywords. The actual model training only needs to label the strong labeled data, which can significantly reduce the data labeling cost.

[0087] In one embodiment, in step S4, the similarity network is trained based on the discriminator to obtain the trained similarity network, including:

[0088] The similarity network is composed of an enumeration layer and a fully connected layer. The enumeration layer is responsible for connecting each pair of picture features extracted by the backbone network, referred to as relationship features, denoted as r. The fully connected layer is responsible for outputting a similarity score for the spliced relationship features. The classification loss of the relationship features is supervised by a binary relationship category label. If a pair of pictures is from the same category, it is the "similar" category, denoted as 1, and vice versa, which is the "not similar" category, denoted as 0. The similarity score can be calculated using the following formula:

[0089] s i,j =P(G(x i ,x j ))=P(r i,j )

[0090] where x i and x j are the ith picture and the jth picture, respectively, G(x i , x j ) is the relationship feature of the image pair formed by the ith picture and the jth picture, and P(r i,j ) is the similarity score between the ith picture and the jth picture.

[0091] In this embodiment, a discriminator is added to the similarity network, which is used to distinguish whether the relationship feature r comes from the base category dataset or the new category dataset. The discriminator can be represented by the following formula:

[0092] d i,j =D(r i,j ) (2)

[0093] where d i,j is the discriminator score, r i,j is the relationship feature of the image pair formed by the ith picture and the jth picture, b represents the base category dataset, and n represents the new category dataset.

[0094] During the training of the similarity network, each iteration includes two steps. First, the similarity network is frozen to minimize the adversarial loss of the discriminator. The calculation formula of the adversarial loss is as follows:

[0095]

[0096] where L D is the adversarial loss, M is the batch size of the input pictures, is the discriminator score when the relationship feature is , and the value is 1, is the relationship feature of the image pair formed by the ith picture and the jth picture in the base category dataset b, is the relationship feature of the image pair formed by the ith picture and the jth picture in the new category dataset n, the discriminator score at the current time, with a value of 0, a relationship feature of an image pair composed of the i-th picture and the j-th picture in the new category dataset n;

[0097] Then the discriminator is frozen, and the classification loss of the similarity network is minimized, and the adversarial loss of the discriminator is maximized. The calculation formula of the classification loss is as follows:

[0098]

[0099] wherein, L G is the classification loss, β is a weighting parameter, which is set to 0.1 through cross-validation, and c i,j is the relationship category label, is the similarity score between the i-th picture and the j-th picture in the basic category dataset b.

[0100] In an embodiment, in step S5, the similarity matrix is used to train the main classifier. The trained main classifier includes:

[0101] In step S51, the similarity matrix is used to calculate the cost function weight and the regularization loss. Specifically, the cost function weight is calculated using the following formula:

[0102]

[0103] wherein, ω c,i represents the cost function weight of the i-th picture, is the number of pictures belonging to the non-kitchen garbage type currently processed, and S c,i,j represents the similarity score between the i-th picture and the j-th picture, and S c,j,i represents the similarity score between the j-th picture and the i-th picture.

[0104] The regularization loss is calculated using the following formula:

[0105]

[0106] wherein, L reg is the regularization loss, s i,j is each element in the similarity matrix S c , h(x i ) is the picture feature of the i-th picture x i , and h(x j ) is the picture feature of the j-th picture x j .

[0107] In step S52, the weighted classification loss is calculated according to the cost function weight. Specifically, the weighted classification loss is calculated using the following formula:

[0108]

[0109] wherein, L cls_ω is the weighted classification loss, is the number of pictures belonging to the non-kitchen garbage type currently processed, is the normalized cost function weight of the i-th picture, x i is the i-th picture, y(x i ) represents the classification score of the picture x i belonging to its corresponding garbage type.

[0110] wherein, the normalized cost function weight of the i-th picture The calculation formula is as follows:

[0111]

[0112] Step S53, the final classification loss is determined according to the regularization loss and the weighted classification loss, and the calculation formula used is:

[0113] L 最终 = L cls_ω + aL reg

[0114] wherein, L 最终 is the final classification loss, L cls_ω is the weighted classification loss, L reg is the regularization loss, and a is a hyperparameter, which is set to 0.1 through cross-validation.

[0115] Step S54, when the final classification loss is the smallest, the trained main classifier is obtained.

[0116] The embodiment of the application also provides a garbage classification method based on weak samples, comprising:

[0117] inputting the garbage picture to be identified into the garbage classification model based on weak samples to obtain the classification score of the garbage picture to be identified belonging to each garbage type; the garbage classification model based on weak samples is the model obtained according to the foregoing embodiment.

[0118] determining the garbage type of the garbage picture to be identified according to the classification score; here, the garbage type with the largest classification score can be selected as the garbage type of the garbage picture to be identified.

[0119] The same inventive concept is adopted as the garbage classification model construction method based on weak samples, and the embodiment also provides a garbage classification model construction device based on weak samples corresponding thereto, Figure 2 a structural block diagram of the garbage classification model construction device based on weak samples according to the embodiment of the application is shown, and the device comprises:

[0120] The basic category dataset obtaining module 21 is configured to perform strong labeling on multiple pictures belonging to the kitchen waste type to obtain a basic category dataset.

[0121] The new category dataset obtaining module 22 is configured to perform weak labeling on multiple pictures belonging to a non-kitchen waste type to obtain a new category dataset. The non-kitchen waste type includes a recyclable waste type, a hazardous waste type and other waste types.

[0122] The picture feature extraction module 23 is configured to extract picture features of the pictures in the basic category dataset and picture features of the pictures in the new category dataset based on the backbone network.

[0123] The similarity network training module 24 is configured to input the picture features of the pictures in the basic category dataset and the picture features of the pictures in the new category dataset into the similarity network, train the similarity network based on the discriminator, and obtain a trained similarity network.

[0124] The main classifier training module 25 is configured to, for each non-kitchen waste type, calculate a similarity between any two pictures belonging to the non-kitchen waste type based on the trained similarity network, and obtain a similarity matrix corresponding to the non-kitchen waste type. The main classifier is trained using the similarity matrix, and a trained main classifier is obtained.

[0125] The model constituting module 26 is configured to constitute a garbage classification model based on weak sample learning by using the backbone network, the trained similarity network and the trained main classifier.

[0126] In this embodiment, by dividing the data into two types of strong labeled data and weak labeled data, the demand for high-quality labeled data in the model training stage is significantly reduced. For weakly labeled data, only labels are needed according to the type, and a large amount of effective data can be quickly obtained by keyword crawling of Internet pictures. The actual model training only needs to label the strong labeled data, which can significantly reduce the data labeling cost.

[0127] In one embodiment, the similarity network training module 24 is further configured to:

[0128] The similarity network is first frozen to minimize the adversarial loss of the discriminator. The calculation formula of the adversarial loss is as follows:

[0129]

[0130] wherein L D is the adversarial loss, M is the batch size of the input pictures, is the discriminator score when the relationship feature is ​a relationship feature of an image pair composed of an i-th picture and a j-th picture in the base category dataset b, a discriminator score when the relationship feature is a relationship feature of an image pair composed of an i-th picture and a j-th picture in the new category dataset n;

[0131] Then the discriminator is frozen, and the similarity network is trained to minimize a relationship classification loss and maximize an adversarial loss of the discriminator, the calculation formula of the classification loss is:

[0132]

[0133] wherein, L G is the classification loss, β is a trade-off parameter, c i,j is a relationship category label, is a similarity score between an i-th picture and a j-th picture in the base category dataset b.

[0134] In an embodiment, the main classifier training module 25 is further configured to:

[0135] calculate a cost function weight and a regularization loss using the similarity matrix;

[0136] calculate a weighted classification loss according to the cost function weight;

[0137] determine a final classification loss according to the regularization loss and the weighted classification loss, and the calculation formula is:

[0138] L 最终 = L cls_ω + αL reg

[0139] wherein, L 最终 is the final classification loss, L cls_ω is the weighted classification loss, L reg is the regularization loss, and α is a hyperparameter;

[0140] When the final classification loss is minimized, the trained main classifier is obtained.

[0141] In an embodiment, the main classifier training module is further configured to:

[0142] calculate a cost function weight using the following formula:

[0143]

[0144] wherein, ω c,i represents a cost function weight of an i-th picture, is the number of pictures belonging to a non-kitchen garbage type currently processed, and S​c,i,j Sij represents a similarity score of the ith picture and the jth picture, S c,j,i Sij represents a similarity score of the jth picture and the ith picture, S

[0145] The regularization loss is calculated by the following formula:

[0146]

[0147] wherein, L reg is the regularization loss, s i,j is each element in the similarity matrix S c , h(x i ) is the picture feature of the ith picture x i , and h(x j ) is the picture feature of the jth picture x j .

[0148] The above merely describes various embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be encompassed in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for constructing a garbage classification model based on weak sample learning, characterized in that, include: Strongly label multiple images belonging to the kitchen waste type to obtain a basic category dataset; The strong annotation refers to labeling the categories of objects contained in the image; Weakly labeling multiple images belonging to non-kitchen waste types yields a new category dataset; the non-kitchen waste types include recyclable waste, hazardous waste, and other waste types. The weak labeling refers to crawling images using a web crawler and then labeling the crawled images with categories. Based on the backbone network, image features of images in the basic category dataset and image features of images in the new category dataset are extracted. The image features of the images in the basic category dataset and the image features of the images in the new category dataset are input into the similarity network. The similarity network is trained based on the discriminator to obtain the trained similarity network. The discriminator is used to distinguish whether the source of the relational features of an image pair consisting of two images is a basic category dataset or a new category dataset. For each type of non-food waste, the similarity between any two images belonging to the non-food waste type is calculated based on the trained similarity network to obtain the similarity matrix corresponding to the non-food waste type; The similarity matrix is ​​used to train the main classifier, resulting in the trained main classifier. The backbone network, the trained similarity network, and the trained master classifier constitute a garbage classification model based on weak sample learning; the backbone network, the trained similarity network, and the trained master classifier are connected in sequence.

2. The method as described in claim 1, characterized in that, The training of the similarity network based on the discriminator includes: First, freeze the similarity network to minimize the adversarial loss of the discriminator. The formula for calculating the adversarial loss is: in, To combat the losses, The batch size of the input images. The relation features are Discriminator score at time The relational features of the image pairs consisting of the i-th and j-th images in the basic dataset b are as follows: The relation features are Discriminator score at time The relational features of the image pair consisting of the i-th image and the j-th image in the new type dataset n; Then, the discriminator is frozen to minimize the relationship classification loss of the similarity network and maximize the adversarial loss of the discriminator. The formula for calculating the classification loss is: in, For classifying losses, To balance the parameters, For relationship category labels, It is the similarity score between the i-th image and the j-th image in the basic category dataset b.

3. The method as described in claim 1, characterized in that, The main classifier is trained using the similarity matrix, and the trained main classifier includes: The cost function weights and regularization loss are calculated using the similarity matrix. Calculate the weighted classification loss based on the weights of the cost function; The final classification loss is determined based on the regularization loss and the weighted classification loss, using the following formula: in, For the final classification loss, For weighted classification loss, For regularization loss, For hyperparameters; When the final classification loss is minimized, the trained main classifier is obtained.

4. The method as described in claim 3, characterized in that, The weights of the cost function are calculated using the following formula: in, This represents the weight of the cost function for the i-th image. This represents the number of images belonging to the non-food waste type currently being processed. Let represent the similarity score between the i-th image and the j-th image. Let represent the similarity score between the j-th image and the i-th image; The regularization loss is calculated using the following formula: in, For regularization loss, Similarity matrix Each element in For the i-th image Image features, For the j-th image Image characteristics.

5. The method as described in claim 3, characterized in that, The weighted classification loss is calculated using the following formula: in, For weighted classification loss, This represents the number of images belonging to the non-food waste type currently being processed. Let be the normalized cost function weights for the i-th image. For the i-th image, Image The classification score corresponding to its waste type.

6. A waste sorting method based on weak samples, characterized in that, include: The garbage images to be identified are input into a garbage classification model based on weak samples to obtain the classification score of each garbage type. The type of waste in the waste image to be identified is determined based on the classification score; The waste classification model based on weak samples is obtained according to any one of claims 1-5.

7. A device for constructing a garbage classification model based on weak sample learning, characterized in that, include: The basic category dataset acquisition module is used to strongly label multiple images belonging to the kitchen waste type to obtain the basic category dataset; The strong annotation refers to labeling the categories of objects contained in the image; The new category dataset acquisition module is used to perform weak labeling on multiple images belonging to the non-kitchen waste type to obtain a new category dataset; the non-kitchen waste type includes recyclable waste type, hazardous waste type, and other waste type; the weak labeling is performed by crawling images and labeling the crawled images with categories; The image feature extraction module is used to extract image features of images in the basic category dataset and image features of images in the new category dataset based on the backbone network. The similarity network training module is used to input the image features of the images in the basic category dataset and the image features of the images in the new category dataset into the similarity network, and train the similarity network based on the discriminator to obtain the trained similarity network. The discriminator is used to distinguish whether the source of the relational features of an image pair consisting of two images is a basic category dataset or a new category dataset. The main classifier training module is used to calculate the similarity between any two images belonging to the non-kitchen waste type based on the trained similarity network for each non-kitchen waste type, and obtain the similarity matrix corresponding to the non-kitchen waste type. The similarity matrix is ​​used to train the main classifier, resulting in the trained main classifier. The model building module is used to construct a garbage classification model based on weak sample learning, consisting of the backbone network, the trained similarity network, and the trained main classifier; the backbone network, the trained similarity network, and the trained main classifier are connected in sequence.

8. The apparatus as claimed in claim 7, characterized in that, The similarity network training module is also used for: First, freeze the similarity network to minimize the adversarial loss of the discriminator. The formula for calculating the adversarial loss is: in, To combat the losses, The batch size of the input images. The relation features are Discriminator score at time The relational features of the image pairs consisting of the i-th and j-th images in the basic dataset b are as follows: The relation features are Discriminator score at time The relational features of the image pair consisting of the i-th image and the j-th image in the new type dataset n; Then, the discriminator is frozen to minimize the relationship classification loss of the similarity network and maximize the adversarial loss of the discriminator. The formula for calculating the classification loss is: in, For classifying losses, To balance the parameters, For relationship category labels, It is the similarity score between the i-th image and the j-th image in the basic category dataset b.

9. The apparatus as claimed in claim 7, characterized in that, The main classifier training module is also used for: The cost function weights and regularization loss are calculated using the similarity matrix. Calculate the weighted classification loss based on the weights of the cost function; The final classification loss is determined based on the regularization loss and the weighted classification loss, using the following formula: in, For the final classification loss, For weighted classification loss, For regularization loss, For hyperparameters; When the final classification loss is minimized, the trained main classifier is obtained.

10. The apparatus as claimed in claim 9, characterized in that, The main classifier training module is also used for: The weights of the cost function are calculated using the following formula: in, This represents the weight of the cost function for the i-th image. This represents the number of images belonging to the non-food waste type currently being processed. Let represent the similarity score between the i-th image and the j-th image. Let represent the similarity score between the j-th image and the i-th image; The regularization loss is calculated using the following formula: in, For regularization loss, Similarity matrix Each element in For the i-th image Image features, For the j-th image Image characteristics.

Images