A deep learning-based spam detection method and system

Abstract

The application relates to a garbage detection method and system based on deep learning, wherein the method comprises the following steps: S1, collecting garbage images; S2, extracting multi-scale features of the garbage images through a backbone network of an improved YOLOv11 model; S3, performing feature fusion on the multi-scale features through a neck network of the improved YOLOv11 model; and S4, detecting the fused features through a detection head of the improved YOLOv11 model to obtain the category of the garbage. The application can effectively detect garbage of different categories, and can especially accurately identify small garbage and easily-confused garbage.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and in particular to a garbage detection method based on deep learning. Background Technology

[0002] With the comprehensive implementation of waste sorting policies, intelligent waste detection technology based on computer vision has become a research hotspot in the industry. YOLO series models are widely used in this field due to their fast detection speed and strong real-time performance; however, existing technologies still have the following core shortcomings in waste detection scenarios:

[0003] Low accuracy in small target detection: There are a lot of small pieces of trash in the garbage scene (such as button batteries, scraps of paper, and small pieces of plastic). The existing YOLO model has insufficient backbone network feature extraction capability and weak semantic information of shallow features, making it difficult to accurately capture the features of small targets, resulting in a high false negative rate.

[0004] Confused waste is poorly identified: Plastic bottles and glass bottles, cardboard and cardboard are similar in visual features but have different category attributes. Pure visual feature extraction models cannot distinguish them, resulting in a high false detection rate.

[0005] Insufficient feature fusion: Existing YOLO models do not incorporate the specific semantic information of garbage classification tasks when processing features, and therefore cannot balance target localization accuracy and class discrimination.

[0006] Lightweight design and performance imbalance: Embedded waste sorting devices have stringent requirements for model size and inference speed. Existing lightweight improvements are mostly simple module replacements, which can easily lead to a decline in feature representation capabilities.

[0007] In summary, existing waste detection methods lack synergistic optimization for small-target waste and the easily confused characteristics of waste, resulting in high rates of missed detection and false detection. Summary of the Invention

[0008] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the existing technology lacks collaborative optimization for small target garbage and the easily confused characteristics of garbage, resulting in a high rate of missed detection and false detection in garbage detection.

[0009] To address the aforementioned technical problems, this invention provides a deep learning-based garbage detection method, comprising:

[0010] Step S1: Collect images of the garbage;

[0011] Step S2: Extract multi-scale features of the garbage image using the backbone network of the improved YOLOv11 model;

[0012] Step S3: Perform feature fusion on the multi-scale features using the neck network of the improved YOLOv11 model;

[0013] Step S4: Detect the fused features using the detection head of the improved YOLOv11 model to obtain the category of garbage.

[0014] In one embodiment of the present invention, the backbone network of the improved YOLOv11 model in step S2 includes:

[0015] Replace the C3k units in the C3k2 module of the backbone network with SACU units, wherein the calculation formula for the SACU unit is:

[0016] ;

[0017] in, Input feature map; Use the Sigmoid activation function to implement semantic guidance gating; , The weights and biases of the 1×1 convolution are used for dimension adjustment; It is a 3×3 depthwise separable convolution used for lightweight feature extraction; , Weights and biases for depthwise separable convolutions; void ratio 3×3 dilated convolutions are used to expand the receptive field; This is an element-wise multiplication method that enables gating control.

[0018] In one embodiment of the present invention, the deep learning-based garbage detection method according to claim 1 is characterized in that: it further includes constructing a total loss function to train the improved YOLOv11 model, so that the improved YOLOv11 model converges, wherein,

[0019] The formula for the total loss function is:

[0020] ;

[0021] in, This is the total loss function; For CIoU loss; Cross-entropy loss for category classification; This represents the loss for cross-layer semantic consistency. For instance-prototype contrastive loss; This represents a loss of prototype diversity. These are the first, second, third, and fourth weighting coefficients.

[0022] In one embodiment of the present invention, the CIoU loss Represented as:

[0023] ;

[0024] in, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Center of the prediction box With real frame The square of the Euclidean distance at the center; The length of the diagonal of the smallest bounding rectangle enclosing the two frames; These are the weighting coefficients; A parameter used to measure the consistency of aspect ratio.

[0025] In one embodiment of the present invention, the construction of the total loss function further includes:

[0026] Building a semantic prototype library ,in, No. A prototype represents a predefined garbage characteristic; This refers to the number of prototypes.

[0027] Constructing semantic vectors: The input image is processed through the backbone network of an improved YOLOv11 model to extract multi-scale visual features, and global visual features are obtained through global average pooling. ;

[0028] calculate With prototype library All semantic prototypes The cosine similarity was calculated, and the weight distribution was obtained by normalization using the Softmax function. :

[0029] According to weight distribution and prototype Constructing semantic vectors .

[0030] In one embodiment of the present invention, the weight distribution Represented as:

[0031] ;

[0032] in, To calculate cosine similarity; This is the adjustment coefficient;

[0033] The semantic vector Represented as:

[0034] .

[0035] In one embodiment of the present invention, the cross-layer semantic consistency loss The construction methods include:

[0036] The input image is processed through the backbone network of the improved YOLOv11 model. Simultaneously, the outputs of the second C3k2 module, the third C3k2 module, and the final output of the backbone network are obtained to construct global visual features corresponding to the three outputs. , And generate corresponding semantic vectors respectively. Based on semantic vectors Calculate the cross-layer semantic consistency loss:

[0037] ;

[0038] in, This is the semantic vector corresponding to the output of the second C3k2 module in the backbone network. The semantic vector corresponding to the output of the third C3k2 module of the backbone network. This refers to the semantic vector corresponding to the multi-scale visual features ultimately output by the backbone network.

[0039] In one embodiment of the present invention, the instance-prototype contrastive loss The methods for constructing tables include:

[0040] Obtain global visual features of the input image global visual features Perform different data augmentations to obtain features and ;

[0041] Calculate separately , With prototype library All semantic prototypes The cosine similarity was calculated, and the weight distribution was obtained by normalization using the Softmax function. and ,based on and Building Instance-Prototype Comparison Loss :

[0042] ;

[0043] in, To calculate the cross-entropy loss.

[0044] In one embodiment of the invention, the prototype diversity loss Represented as:

[0045] ;

[0046] in, For semantic prototype library The Middle The and the first A prototype, To calculate cosine similarity, These are boundary values.

[0047] To address the aforementioned technical problems, this invention provides a deep learning-based garbage detection system, comprising:

[0048] Acquisition module: Acquires images of garbage;

[0049] Extraction module: Extracts multi-scale features of the garbage image through a backbone network of an improved YOLOv11 model;

[0050] Feature fusion module: performs feature fusion on the multi-scale features using the neck network of the improved YOLOv11 model;

[0051] Detection module: The detection head of the improved YOLOv11 model is used to detect the fused features and obtain the category of garbage.

[0052] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0053] The deep learning-based garbage detection method described in this invention improves the YOLOv11 model by replacing the second C3k unit in the cascaded C3k units of the backbone network's C3k2 module with a SACU unit. The SACU unit can effectively detect small pieces of garbage.

[0054] This invention constructs a total loss function (integrating CIoU loss, cross-layer semantic consistency loss, instance-prototype contrast loss, and prototype diversity loss), and effectively trains the improved YOLOv11 model through the total loss function, thereby enhancing the ability to identify easily confused spam and effectively reducing the false detection rate.

[0055] This invention has strong generalization ability. By improving the YOLOv11 model, it reduces the sensitivity to changes in lighting and scene, and still maintains high detection accuracy in complex scenes. Attached Figure Description

[0056] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2 This is a schematic diagram of the YOLOv11 model structure in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the improved C3k2 module structure in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the SACU unit structure in an embodiment of the present invention. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0062] Example 1

[0063] Reference Figure 1 As shown, this invention relates to a deep learning-based garbage detection method, comprising:

[0064] Step S1: Collect images of the garbage;

[0065] Step S2: Extract multi-scale features of the garbage image using the backbone network of the improved YOLOv11 model;

[0066] Step S3: Perform feature fusion on the multi-scale features using the improved YOLOv11 model's neck network;

[0067] Step S4: Detect the fused features using the detection head of the improved YOLOv11 model to obtain the garbage category.

[0068] The following is a detailed description of this embodiment:

[0069] This invention employs an improved YOLOv11 model, and the specific technical solution is as follows:

[0070] 1. Overall Model Architecture Design

[0071] The improved YOLOv11 model in this embodiment includes: a backbone network, a self-supervised semantic prototype generation unit (AESG unit), a hierarchical optimized detection head, and a multi-objective collaborative loss function, as detailed below:

[0072] (1) Backbone network: For the specific network structure of the YOLOv11 model, please refer to Figure 2 Since the YOLOv11 model is existing technology, its structure will not be described in detail in this embodiment. This embodiment improves the C3k2 module of the backbone network in the YOLOv11 model (see [link to documentation] for details). Figure 3This embodiment introduces semantically aware convolutional units (SACUs) to replace the original C3k units in the C3k2 module (C3k units are existing technology and will not be described in detail here), thereby enhancing the semantic representation of small target features. Since the number of C3k units in the C3k2 module of the YOLOv11 model can be adjusted, this embodiment sets the number of C3k units to two, and replaces the second C3k unit in the two cascaded C3k units with an SACU unit. It should be noted that since both the backbone and neck networks in the YOLOv11 model contain C3k2 modules, this embodiment only improves the C3k2 module in the backbone network. In other embodiments, the C3k2 modules of both the backbone and neck networks can be improved simultaneously.

[0073] (2) AESG unit (used to construct the total loss function): realizes label-free dynamic semantic vector generation and cross-modal deep fusion;

[0074] (3) Layered optimization detection head: Anchor frames are customized for different sizes of waste to achieve accurate layered detection of small, medium and large waste;

[0075] (4) Multi-objective collaborative loss function: Construct a total loss function that integrates CIoU loss, cross-layer semantic consistency loss, instance-prototype contrast loss, and prototype diversity loss to achieve effective training of the improved YOLOv11 model.

[0076] 2. Improved backbone network (including SACU units)

[0077] This embodiment embeds SACU units into the residual paths of the original C3k2 module in the YOLOv11 model (for the backbone network), replacing the C3k units to achieve effective detection of small targets (small pieces of trash). The SACU unit is as follows:

[0078] The SACU unit is used to sequentially pass the input feature map through a 1×1 convolution and a Sigmoid activation function to obtain a first feature map, pass the input feature map through a 3×3 depthwise separable convolution to obtain a second feature map, pass the input feature map through a 3×3 dilated convolution to obtain a third feature map, perform element-wise multiplication between the first and second feature maps to obtain the element-wise multiplication result, and then perform residual concatenation between the element-wise multiplication result and the third feature map to obtain the output of the SACU unit.

[0079] Please see Figure 4 The processing procedure of the SACU unit in this embodiment is as follows:

[0080]

[0081] in, Input feature map; Use the Sigmoid activation function to implement semantic guidance gating; , The weights and biases of the 1×1 convolution are used for dimension adjustment; It is a 3×3 depthwise separable convolution used for lightweight feature extraction; , Weights and biases for depthwise separable convolutions; void ratio 3×3 dilated convolutions are used to expand the receptive field; This is an element-wise multiplication method that enables gating control.

[0082] 3. Self-supervised semantic prototype generation unit (AESG)

[0083] Constructing the final total loss function using AESG units is the core method for resolving "training-inference inconsistency" (i.e., encountering irrelevant features during model training that were not encountered during actual inference). To improve the model's generalization ability, even if irrelevant features are encountered during actual inference, the total loss function constructed using AESG units in this embodiment ensures that the finally trained model can still correctly identify these previously unused features. AESG units do not rely on any category labels. The specific implementation of the AESG unit is as follows:

[0084] 3.1 Semantic Prototype Library Construction

[0085] Initialize a learnable semantic prototype library in the network. ,in, No. A prototype represents a predefined garbage characteristic; This represents the number of prototypes.

[0086] Each prototype It is a 128-dimensional vector representing the potential semantic concepts in the garbage data, such as garbage being transparent, garbage being cylindrical, garbage being compressible, etc. The specifics can be predefined according to the actual situation.

[0087] This embodiment will have a prototype quantity Set the number of waste subcategories to 2-3 times the number of subcategories. In this embodiment, the number of waste subcategories can be set to 12, therefore... .

[0088] 3.2 Semantic Vector Generation

[0089] (1) After the input image is processed by the backbone network to extract multi-scale visual features, global visual features are obtained by global average pooling. Its dimensions are the same as the aforementioned prototype. The vectors are identical, both being 128-dimensional.

[0090] (2) Calculation With prototype library All semantic prototypes The cosine similarity was calculated, and the weight distribution was obtained by normalization using the Softmax function. ;

[0091] (3) Based on the weight distribution and prototype Constructing semantic vectors The formula is as follows:

[0092]

[0093]

[0094] in, To calculate cosine similarity; This is an adjustment coefficient (0.1 in this embodiment) used to adjust the smoothness of the weight distribution.

[0095] 3.3 Constructing the total loss function

[0096] This embodiment also includes constructing a total loss function to train the improved YOLOv11 model, causing the improved YOLOv11 model to converge. The formula for the total loss function is as follows:

[0097]

[0098] in, This is the total loss function; For CIoU loss; Cross-entropy loss for category classification; This represents the loss for cross-layer semantic consistency. For instance-prototype contrastive loss; This represents a loss of prototype diversity. These are the first, second, third, and fourth weighting coefficients. In this embodiment... , , , .

[0099] Furthermore, CIoU loss The formula is:

[0100]

[0101] in, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Center of the prediction box With real frame The square of the Euclidean distance at the center; The length of the diagonal of the smallest bounding rectangle enclosing the two frames; These are the weighting coefficients; A parameter used to measure the consistency of aspect ratio.

[0102] Furthermore, instance-prototype contrast loss The construction method is as follows:

[0103] The input image is processed through the aforementioned backbone network and global average pooling to obtain global visual features. global visual features Perform two independent data augmentations (such as mosaic augmentation, random flipping, cropping, etc.) to obtain features. and Based on features and Generate prototype weight distributions respectively and ;based on and Building Instance-Prototype Comparison Loss :

[0104]

[0105] in, The cross-entropy loss is used to ensure that the distributions of the two are consistent.

[0106] Furthermore, prototype diversity loss To encourage the dispersed distribution of prototypes in the semantic space, the formula is:

[0107]

[0108] in, For semantic prototype library The Middle The and the first A prototype, To calculate cosine similarity, The boundary value (-0.5 in this embodiment) allows for moderate similarity between prototypes but avoids complete overlap.

[0109] Furthermore, cross-layer semantic consistency loss This is used to enhance the semantic and visual correlation. First, the global visual features corresponding to the outputs of the shallow layer (corresponding to the second C3k2 module of the backbone network), the middle layer (corresponding to the third C3k2 module of the backbone network), and the deep layer (corresponding to the C2PSA module of the backbone network) of the backbone network are extracted. ( ), and generate corresponding semantic vectors respectively. (Right now , , );

[0110] Based on semantic vectors The formula for calculating cross-layer semantic consistency loss is as follows:

[0111]

[0112] in, This is the semantic vector corresponding to the output of the second C3k2 module in the backbone network. The semantic vector corresponding to the output of the third C3k2 module of the backbone network. This refers to the semantic vector corresponding to the multi-scale visual features output by the backbone network. It should be noted that the second and third C3k2 modules of the backbone network are counted starting from the input direction of the backbone network; see the documentation for details. Figure 2 .

[0113] 4. Layered optimization of the detection head

[0114] (1) In this embodiment, after identifying garbage in the improved YOLOv11 model, hierarchical anchor box optimization can be performed. Specifically, the box results identified by the improved YOLOv11 model are clustered into three detection layers using K-means to customize anchor boxes, so as to achieve re-classification of garbage. The purpose is to detect whether it really belongs to small garbage, medium garbage, or large garbage, as follows:

[0115] Shallow detection layer: responsible for small waste ≤5cm, with anchor frames of (0.05,0.03), (0.08,0.06), and (0.12,0.09);

[0116] Middle layer detection layer: responsible for medium-sized waste of 5-20cm, with anchor frames of (0.2,0.15), (0.3,0.25), and (0.4,0.35);

[0117] Deep detection layer: responsible for large waste ≥20cm, with anchor frames of (0.6,0.5), (0.8,0.7), and (1.0,0.9).

[0118] Example 2

[0119] This embodiment provides a deep learning-based garbage detection system, including:

[0120] Acquisition module: Acquires images of garbage;

[0121] Extraction module: Extracts multi-scale features of the garbage image through a backbone network of an improved YOLOv11 model;

[0122] Feature fusion module: performs feature fusion on the multi-scale features using the neck network of the improved YOLOv11 model;

[0123] Detection module: The detection head of the improved YOLOv11 model is used to detect the fused features and obtain the category of garbage.

[0124] Example 3

[0125] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the deep learning-based garbage detection method described in Embodiment 1.

[0126] Example 4

[0127] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the deep learning-based garbage detection method described in Embodiment 1.

[0128] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0129] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A garbage detection method based on deep learning, characterized in that, include: Step S1: Collect images of the garbage; Step S2: Extract multi-scale features of the garbage image using the backbone network of the improved YOLOv11 model; Step S3: Perform feature fusion on the multi-scale features using the neck network of the improved YOLOv11 model; Step S4: Detect the fused features using the detection head of the improved YOLOv11 model to obtain the category of garbage.

2. The deep learning-based garbage detection method according to claim 1, characterized in that: The backbone network of the improved YOLOv11 model in step S2 includes: Replace the C3k units in the C3k2 module of the backbone network with SACU units, wherein the calculation formula for the SACU unit is: ； in, Input feature map; Use the Sigmoid activation function to implement semantic guidance gating; , The weights and biases of the 1×1 convolution are used for dimension adjustment; It is a 3×3 depthwise separable convolution used for lightweight feature extraction; , Weights and biases for depthwise separable convolutions; void ratio 3×3 dilated convolutions are used to expand the receptive field; This is an element-wise multiplication method that enables gating control.

3. The deep learning-based garbage detection method according to claim 1, characterized in that: This also includes constructing a total loss function to train the improved YOLOv11 model, causing the improved YOLOv11 model to converge, wherein... The formula for the total loss function is: ； in, This is the total loss function; For CIoU loss; Cross-entropy loss for category classification; This represents the loss for cross-layer semantic consistency. For instance-prototype contrastive loss; This represents a loss of prototype diversity. These are the first, second, third, and fourth weighting coefficients.

4. The deep learning-based garbage detection method according to claim 3, characterized in that: The CIoU loss Represented as: ； in, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Center of the prediction box With real frame The square of the Euclidean distance at the center; The length of the diagonal of the smallest bounding rectangle enclosing the two frames; These are the weighting coefficients; A parameter used to measure the consistency of aspect ratio.

5. The deep learning-based garbage detection method according to claim 3, characterized in that: The construction of the total loss function also includes: Building a semantic prototype library ,in, No. A prototype represents a predefined garbage characteristic; This refers to the number of prototypes. Constructing semantic vectors: The input image is processed through the backbone network of an improved YOLOv11 model to extract multi-scale visual features, and global visual features are obtained through global average pooling. ; calculate With prototype library All semantic prototypes The cosine similarity was calculated, and the weight distribution was obtained by normalization using the Softmax function. : According to weight distribution and prototype Constructing semantic vectors .

6. The deep learning-based garbage detection method according to claim 5, characterized in that: The weight distribution Represented as: ； in, To calculate cosine similarity; This is the adjustment coefficient; The semantic vector Represented as: 。 7. The deep learning-based garbage detection method according to claim 5, characterized in that: The cross-layer semantic consistency loss The construction methods include: The input image is processed through the backbone network of the improved YOLOv11 model. Simultaneously, the outputs of the second C3k2 module, the third C3k2 module, and the final output of the backbone network are obtained to construct global visual features corresponding to the three outputs. , And generate corresponding semantic vectors respectively. Based on semantic vectors Calculate the cross-layer semantic consistency loss: ； in, This is the semantic vector corresponding to the output of the second C3k2 module in the backbone network. The semantic vector corresponding to the output of the third C3k2 module of the backbone network. This refers to the semantic vector corresponding to the multi-scale visual features ultimately output by the backbone network.

8. The deep learning-based garbage detection method according to claim 5, characterized in that: The instance-prototype contrast loss The methods for constructing tables include: Obtain global visual features of the input image global visual features Perform different data augmentations to obtain features and ; Calculate separately , With prototype library All semantic prototypes The cosine similarity was calculated, and the weight distribution was obtained by normalization using the Softmax function. and ,based on and Building Instance-Prototype Comparison Loss : ； in, To calculate the cross-entropy loss.

9. The deep learning-based garbage detection method according to claim 3, characterized in that: The prototype diversity loss Represented as: ； in, For semantic prototype library The Middle The and the first A prototype, To calculate cosine similarity, These are boundary values.

10. A deep learning-based garbage detection system, characterized in that, include: Acquisition module: Acquires images of garbage; Extraction module: Extracts multi-scale features of the garbage image through a backbone network of an improved YOLOv11 model; Feature fusion module: performs feature fusion on the multi-scale features using the neck network of the improved YOLOv11 model; Detection module: The detection head of the improved YOLOv11 model is used to detect the fused features and obtain the category of garbage.

Images