Model training method, and garbage detection method and device

Abstract

The embodiment of the application provides a model training method, a garbage detection method and device, relates to the technical field of target detection, and the model training method comprises the following steps: training a first model by using a first training set; setting network parameters of a backbone network of a second model as network parameters of the backbone network of the first model; training the second model by using a second training set; setting network parameters of the backbone network of a third model as the network parameters of the backbone network of the first model, setting network parameters of a neck network of the third model as network parameters of the neck network of the second model, setting network parameters of a head network of the third model as network parameters of the head network of the second model; and training the third model by using a third training set, so that the accuracy rate of the third model in identifying a target object category reaches a third threshold value. The generalization performance of the model is improved, and the accuracy of a detection result is improved in the case that the training data is limited.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a model training method and a garbage detection method and device. BACKGROUND

[0002] Garbage classification is a trend of social development. In a high point video monitoring scene of a tower, garbage detection can improve the efficiency of garbage classification. At present, a network model based on deep learning is generally used to detect target objects in an image. Factors affecting the detection effect of the network model include a training set used when the network model is trained.

[0003] However, the training set for garbage detection in the prior art is limited, and the training set for garbage detection in a high point video monitoring scene is even less, resulting in poor training effect of the garbage detection model, thereby leading to poor garbage detection accuracy.

[0004] It can be seen that the prior art has the problem of poor garbage detection accuracy caused by poor training effect of the garbage detection model. SUMMARY

[0005] The embodiments of the present application provide a model training method, a garbage detection method and device to solve the problem of poor garbage detection accuracy caused by poor training effect of the garbage detection model in the prior art.

[0006] The embodiments of the present application provide a model training method, which comprises:

[0007] training a first model using a first training set to make the accuracy of the first model in detecting target objects reach a first threshold;

[0008] setting the network parameters of the backbone network of a second model as the network parameters of the backbone network of the first model;

[0009] training the second model using a second training set to make the accuracy of the second model in identifying target object categories reach a second threshold;

[0010] setting the network parameters of the backbone network of a third model as the network parameters of the backbone network of the first model, setting the network parameters of the neck network of the third model as the network parameters of the neck network of the second model, and setting the network parameters of the head network of the third model as the network parameters of the head network of the second model;

[0011] training the third model using a third training set to make the accuracy of the third model in identifying target object categories reach a third threshold.

[0012] Optionally, before the training the second model using the second training set, the method further comprises:

[0013] obtaining a pre-acquired garbage sample image;

[0014] performing data enhancement processing on the garbage sample image to obtain the second training set.

[0015] Optionally, before the training the third model using the third training set, the method further comprises:

[0016] obtaining a pre-acquired garbage sample image of a preset shooting point;

[0017] performing data enhancement processing on the garbage sample image of the preset shooting point to obtain the third training set.

[0018] Optionally, the data enhancement processing comprises at least one of splicing processing, hue saturation value (HSV) processing, offset processing, scaling processing and flipping processing.

[0019] Optionally, the training the third model using the third training set to make the accuracy rate of the third model in identifying a target object category reach a third threshold value comprises:

[0020] iterating network parameters of a neck network and network parameters of a head network in the third model until a loss function of data output by the head network in the third model reaches a target threshold value, wherein the loss function comprises a confidence loss, a classification loss and a coordinate regression loss.

[0021] Optionally, the second training set and the third training set each comprise positive sample data and negative sample data, the positive sample data comprises at least one of a scattered household garbage sample image, a building material pile garbage sample image and a household garbage pile sample image, and the negative sample data comprises at least one of a non-garbage pile sample image, a road pit pile sample image, a garbage can sample image, a garbage bin sample image, a truck sample image, a garbage truck sample image, a road manhole cover sample image and a masonry pile sample image.

[0022] The embodiment of the application further provides a garbage detection method, which comprises:

[0023] performing preprocessing on an obtained target image, the target image being an image obtained from a video shot from a preset shooting point on a tower, and the preprocessing comprising scaling processing and normalization processing;

[0024] inputting the target image into a pre-trained third model to generate a detection result;

[0025] An alarm feedback is performed according to the detection result.

[0026] The embodiment of the present application also provides a model training device, and the device comprises:

[0027] A first training module is configured to train a first model by using a first training set, so that the accuracy of the first model in detecting a target object reaches a first threshold;

[0028] A first setting module is configured to set the network parameters of a backbone network of a second model as the network parameters of the backbone network of the first model;

[0029] A second training module is configured to train the second model by using a second training set, so that the accuracy of the second model in identifying a target object category reaches a second threshold;

[0030] A second setting module is configured to set the network parameters of a backbone network of a third model as the network parameters of the backbone network of the first model, set the network parameters of a neck network of the third model as the network parameters of the neck network of the second model, and set the network parameters of a head network of the third model as the network parameters of the head network of the second model;

[0031] A third training module is configured to train the third model by using a third training set, so that the accuracy of the third model in identifying a target object category reaches a third threshold.

[0032] Optionally, the device further comprises:

[0033] A first acquisition module is configured to acquire a pre-acquired garbage sample image;

[0034] A first processing module is configured to perform data enhancement processing on the garbage sample image, so as to obtain the second training set.

[0035] Optionally, the device further comprises:

[0036] A second acquisition module is configured to acquire a pre-acquired garbage sample image of a preset shooting point;

[0037] A second processing module is configured to perform data enhancement processing on the garbage sample image of the preset shooting point, so as to obtain the third training set.

[0038] Optionally, the data enhancement processing comprises at least one of splicing processing, hue saturation brightness (HSV) processing, offset processing, scaling processing and flipping processing.

[0039] Optionally, the third training module comprises:

[0040] An iteration submodule is configured to iterate network parameters of the neck network and network parameters of the head network in the third model until a loss function of the head network output data in the third model reaches a target threshold, wherein the loss function comprises a confidence loss, a classification loss and a coordinate regression loss.

[0041] Optionally, the second training set and the third training set each comprise positive sample data and negative sample data, the positive sample data comprises at least one of scattered household garbage sample images, building material pile garbage sample images and household garbage pile sample images, and the negative sample data comprises at least one of non-garbage pile sample images, road pit pile sample images, garbage can sample images, garbage bin sample images, truck sample images, garbage truck sample images, road manhole cover sample images and masonry pile sample images.

[0042] The embodiment of the present application also provides a garbage detection device, the device comprises:

[0043] A preprocessing module is configured to pre-process a target image, the target image being an image acquired from a video captured from a preset shooting point on a tower, and the pre-processing comprising scaling processing and normalization processing;

[0044] A generation module is configured to input the target image into a third model pre-trained to generate a detection result;

[0045] A feedback module is configured to perform alarm feedback according to the detection result.

[0046] In the embodiment of the present application, a multi-level transfer learning technology is adopted in the model training process, first, the feature extraction capability of the first model is pre-trained by means of a large batch of first training sets in a general scene, then the network parameters of the backbone network of the trained first model are transferred to the backbone network of the second model, and the feature recognition capability of the second model is trained based on a batch of second training sets in a certain specific scene, then the network parameters of the backbone network of the trained first model are transferred to the backbone network of the third model, and the network parameters corresponding to the neck network and the head network of the trained second model are transferred to the neck network and the head network of the third model, and the specific angle feature recognition capability of the third model is trained based on a small batch of third training sets in a specific angle of the current specific scene. In this way, the generalization performance of the model is improved, and in the case of limited training data, good training effect can also be achieved, and the accuracy of the detection result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is one of the flowcharts illustrating the model training method provided in this embodiment of the invention;

[0049] Figure 2 This is the second flowchart illustrating the model training method provided in this embodiment of the invention;

[0050] Figure 3 This is a schematic diagram of the ELAN structure in the backbone network provided in this embodiment of the invention;

[0051] Figure 4 This is one of the flowcharts of the waste detection method provided in the embodiments of the present invention;

[0052] Figure 5 This is a second schematic flowchart of the waste detection method provided in this embodiment of the invention;

[0053] Figure 6 This is a schematic diagram of the structure of the model training device provided in an embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram of the structure of the waste detection device provided in an embodiment of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such usage can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, the first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0057] Please see Figure 1 , Figure 1 This is one of the flowcharts illustrating the model training method provided in this embodiment of the invention. For example... Figure 1 As shown in the figure, an embodiment of the present invention provides a model training method, which includes the following steps:

[0058] Step 101: Train the first model using the first training set so that the accuracy of the first model in detecting target objects reaches the first threshold.

[0059] The first training set, i.e., the general image domain, consists of millions of data points from general image scenes. The first model can be an open-source model (such as the YOLO series models). The network structure of the first model mainly includes a backbone network, a neck network, and a head network. Based on the millions of general images in the first training set, the first model is trained until its accuracy in detecting target objects reaches a first threshold. This first threshold can be determined based on actual conditions and is not limited here. If the accuracy of the first model in detecting target objects reaches the first threshold, then the first model is considered applicable to target detection. In other words, the first model extracts features from the input image data to obtain superior performance in detecting target objects within the image data. Thus, during the training process of the first model, the large amount of data in the first training set, being general images, facilitates the updating and iteration of the backbone network, thereby improving the accuracy of target object detection.

[0060] Step 102: Set the network parameters of the backbone network of the second model to the network parameters of the backbone network of the first model;

[0061] The first model is primarily used to detect target objects in images. For example, it identifies target objects in an image that differ significantly from their surroundings. These target objects can be people, plants, animals, buildings, or litter. The second model is mainly used to identify the category of the target object. For example, in the application scenario of identifying litter, it identifies target objects in an image that differ significantly from their surroundings as litter, and identifies the category of the litter as household waste, building material waste, or medical waste.

[0062] The second model uses the same network structure as the first model, meaning it also includes a backbone network, a neck network, and a head network. The backbone network parameters of the first model are obtained when the accuracy of the first model in detecting target objects reaches a first threshold. Therefore, by setting the backbone network parameters of the second model to those of the first model, the second model also exhibits superior performance in detecting target objects in images. Thus, during the training of the second model, the network parameters of the replaced backbone network are frozen, and the parameters of the neck and head networks in the second model are iterated based on the second training set in subsequent step 103. This improves the training efficiency of the second model.

[0063] Step 103: Train the second model using the second training set so that the accuracy of the second model in recognizing the target object category reaches the second threshold.

[0064] In this embodiment of the invention, the application of a target model trained by the model training method provided by the present invention to garbage detection is used as an example for illustrative explanation.

[0065] The second training set, the general garbage domain, consists of training data comprised of typical garbage samples. Furthermore, the pixel dimensions of the target garbage in these typical garbage samples can range from 20*20 (extremely small targets), 30*30 (normal small targets), 50*50 to 150*150 (regular targets), to 600*600 (extra-large targets). The categories of target garbage in these typical garbage samples can include scattered household waste, piles of building materials, and large piles of household waste. Thus, the second training set includes target garbage of multiple scales and categories, enhancing the model's generalization ability.

[0066] Typical junk sample data usually consists of tens of thousands of images. Obviously, in specific domains, the amount of sample data in the general junk domain is less than the amount of sample data in the general image domain.

[0067] By setting the backbone network parameters of the second model to those of the first model in step 102, the training process of the backbone network in the second model is essentially transferred to the training process of the first model using the first training set in step 101. When the accuracy of the second model in identifying target object categories reaches a second threshold, it is considered that the second model can be applied to identify target object categories. In other words, the second model performs better in feature extraction from the input image data to determine the category of the target object in the image data. Thus, during the training of the second model, the backbone network parameters obtained after transfer training can be directly utilized. Only the network parameters of the neck and head networks in the second model need to be iterated. Even with a small amount of sample data in the second training set, a good training effect can still be achieved, improving the accuracy of identifying target object categories.

[0068] Step 104: Set the network parameters of the backbone network of the third model to the network parameters of the backbone network of the first model, set the network parameters of the neck network of the third model to the network parameters of the neck network of the second model, and set the network parameters of the head network of the third model to the network parameters of the head network of the second model.

[0069] The third model is primarily used to identify the category of target objects at preset shooting locations, such as the middle to the top of a tower. In one example, a camera can be placed at the top of the tower. By acquiring images captured by this camera, the third model can be used to identify the category of target objects in the images. Preferably, the camera is a high-definition camera to obtain high-definition video, facilitating subsequent feature extraction and improving detection accuracy.

[0070] The third model uses the same network structure as the second model, meaning it also includes a backbone network, a neck network, and a head network. The backbone network parameters of the first model are obtained when the accuracy of the first model in detecting target objects reaches a first threshold. Therefore, by setting the backbone network parameters of the third model to those of the first model, the third model also exhibits superior performance in detecting target objects in images. Furthermore, the neck and head networks of the second model have network parameters obtained when the accuracy of the second model in identifying target object categories reaches a second threshold. Therefore, by setting the neck network parameters of the third model to those of the second model, and similarly, setting the head network parameters of the third model to those of the second model, the third model also exhibits superior performance in identifying target object categories. Thus, during the training of the third model, the network parameters of the replaced backbone network are frozen, and the parameters of the neck and head networks in the third model are iterated based on the third training set in subsequent step 105. This improves the training efficiency of the third model.

[0071] Step 105: Train the third model using the third training set so that the accuracy of the third model in identifying the target object category reaches the third threshold.

[0072] In waste detection applications, the third training set, namely the waste domain at preset shooting locations (e.g., mid-to-high points on a steel tower), consists of waste sample data from these preset shooting locations. This sample data is significantly smaller than regular waste sample data. It typically comprises several thousand video surveillance images of mid-to-high points on a steel tower, containing waste samples. Furthermore, the pixel size of the target waste in these samples can range from 20*20 (extremely small targets) to 30*30 (normal small targets). The categories of target waste can include scattered household waste, construction material piles, and general household waste piles. This multi-scale, multi-category inclusion of target waste in the third training set enhances the model's generalization ability.

[0073] Clearly, in a more specific segment, the amount of sample data in the garbage domain at the preset shooting location is less than the amount of sample data in the general garbage domain. The differences between the garbage sample data at the preset shooting location and the regular garbage sample data include parameters such as viewing angle, resolution, brightness, and saturation.

[0074] By setting the network parameters of the backbone network of the third model to those of the backbone network of the first model in step 104, the training process of the backbone network in the third model is essentially transferred to the process of training the first model using the first training set in step 101. Similarly, setting the network parameters of the neck and head networks of the third model to those of the neck and head networks of the second model is equivalent to transferring the training process of most of the neck network and a small portion of the head network in the third model to the process of training the second model using the second training set in step 103. When training the third model using the third training set, the network parameters of the neck network in the third model are fine-tuned, while the network parameters of the head network in the third model are significantly updated to ensure that the accuracy of the third model in recognizing the target object category at the preset shooting point reaches the third threshold. If the accuracy of the third model in identifying the target object category at the preset shooting point reaches the third threshold, then the third model is considered to be applicable to identifying the target object category at the preset shooting point. In other words, the third model performs feature extraction on the input image data of the preset shooting point to determine the target object category in the image data of the preset shooting point, and has a better performance.

[0075] In this way, during the training of the third model, the network parameters of the backbone network obtained after transfer training can be directly used. It is only necessary to further iterate the network parameters of the neck and head networks in the third model based on the network parameters of the neck and head networks in the second model. Even with less sample data in the third training set, a good training effect can still be achieved, which improves the accuracy of identifying the target object category at the preset shooting point.

[0076] In this embodiment, a multi-level transfer learning technique is employed during model training. First, the feature extraction capabilities of the first model are pre-trained using a large batch of training data from a general scenario. Then, the network parameters of the backbone network in the trained first model are transferred to the backbone network of the second model. Next, the feature recognition capabilities of the second model are trained using a medium batch of training data from a specific scenario. Then, the network parameters of the backbone network in the trained first model are transferred to the backbone network of the third model, and the network parameters corresponding to the neck and head networks in the trained second model are transferred to the neck and head networks of the third model. Finally, the feature recognition capabilities of the third model from a specific perspective in the current specific scenario are trained using a small batch of training data from a specific perspective. This improves the model's generalization performance, achieving good training results even with limited training data, and enhancing the accuracy of the detection results.

[0077] In cases where the training data in the second training set is limited, data augmentation can be used to expand the training data in the second training set, as detailed below:

[0078] Optionally, before training the second model using the second training set, the method further includes:

[0079] Acquire images of pre-collected garbage samples;

[0080] The garbage sample images are subjected to data augmentation processing to obtain the second training set.

[0081] In this embodiment, taking a waste detection application scenario as an example, data augmentation processing is performed on the acquired N1 waste sample images to obtain processed N2 waste sample images. The second training set includes these N2 waste sample images, where N2 is greater than N1, and both N1 and N2 are positive integers. The data augmentation processing includes at least one of the following: stitching processing, hue saturation value (HSV) processing, offset processing, scaling processing, and flipping processing.

[0082] In one example, four garbage sample images can be randomly cropped and scaled to create a single garbage sample image with a certain probability. This combined image is then used as training data in the second training set. The probability hyperparameter for the stitching process can be set to 1. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0083] In another example, the chroma, saturation, and brightness of the original garbage sample image can be randomly adjusted with a certain probability to obtain a processed garbage sample image. The original garbage sample image before processing and the processed garbage sample image are then used as training data in a second training set. Specifically, the probability hyperparameter corresponding to chroma can be set to 0.015, the probability hyperparameter corresponding to saturation can be set to 0.7, and the probability hyperparameter corresponding to brightness can be set to 0.4. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0084] In another example, the original garbage sample image can be translated with a certain probability to obtain a processed garbage sample image. The original garbage sample image and the processed garbage sample image are then used as training data in a second training set. The probability hyperparameter corresponding to the offset processing can be set to 0.2. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0085] In another example, the original garbage sample image can be scaled with a certain probability to obtain a processed garbage sample image. The original garbage sample image and the processed garbage sample image are then used as training data in a second training set. The probability hyperparameter corresponding to the scaling operation can be set to 0.5. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0086] In another example, the original garbage sample image can be flipped horizontally with a certain probability to obtain a processed garbage sample image. The original garbage sample image and the processed garbage sample image are then used as training data in a second training set. The probability hyperparameter corresponding to the horizontal flipping operation can be set to 0.5. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0087] In cases where the training data in the third training set is limited, data augmentation can be used to expand the training data in the third training set, as detailed below:

[0088] Optionally, before training the third model using the third training set, the method further includes:

[0089] Acquire pre-collected images of garbage samples at preset shooting locations;

[0090] Data augmentation processing is performed on the garbage sample images at the preset shooting locations to obtain the third training set.

[0091] In this embodiment, taking a garbage detection application scenario at preset shooting locations as an example, data augmentation processing is performed on the acquired M1 garbage sample images at preset shooting locations to obtain processed M2 garbage sample images at preset shooting locations. The third training set includes these M2 garbage sample images at preset shooting locations, where M2 is greater than M1, and both M1 and M2 are positive integers. The data augmentation processing includes at least one of the following: stitching processing, hue saturation value (HSV) processing, offset processing, scaling processing, and flipping processing.

[0092] In one example, four garbage sample images from preset shooting locations can be randomly cropped and scaled to create a single garbage sample image from the same preset shooting location, with a certain probability. These four images and the resulting composite image are then used as training data in a third training set. The probability hyperparameter for the stitching process can be set to 1. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0093] In another example, the chroma, saturation, and brightness of the original garbage sample images at preset shooting locations can be randomly adjusted with a certain probability to obtain processed garbage sample images at the preset shooting locations. The original garbage sample images at the preset shooting locations before processing and the processed garbage sample images at the preset shooting locations are used as training data in a third training set. Specifically, the probability hyperparameter corresponding to chroma can be set to 0.015, the probability hyperparameter corresponding to saturation can be set to 0.7, and the probability hyperparameter corresponding to brightness can be set to 0.4. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0094] In another example, the original garbage sample images at preset shooting locations can be translated with a certain probability to obtain processed garbage sample images at the preset shooting locations. The original and processed garbage sample images at the preset shooting locations are then used as training data in a third training set. The probability hyperparameter corresponding to the offset processing can be set to 0.2. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0095] In another example, the original garbage sample images at preset shooting locations can be scaled with a certain probability to obtain processed garbage sample images at the preset shooting locations. The original and processed garbage sample images at the preset shooting locations are then used as training data in a third training set. The probability hyperparameter corresponding to the scaling operation can be set to 0.5. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0096] In another example, the original garbage sample images at preset shooting locations can be flipped horizontally with a certain probability to obtain processed garbage sample images at the preset shooting locations. The original and processed garbage sample images at the preset shooting locations are then used as training data in a second training set. The probability hyperparameter corresponding to the horizontal flipping operation can be set to 0.5. This expands the training dataset, improves the model's generalization performance, and thus enhances the accuracy of the detection results.

[0097] Optionally, training the third model using a third training set to achieve a third threshold in the accuracy of the third model in identifying target object categories includes:

[0098] The network parameters of the neck network and the head network in the third model are iterated until the loss function of the output data of the head network in the third model reaches the target threshold. The loss function includes confidence loss, classification loss and coordinate regression loss.

[0099] In this embodiment, such as Figure 2 As shown, the training process for any of the first, second, or third models may include the following steps:

[0100] Data Collection and Cleaning: Training data collection is divided into real-world scene samples from the same source and other samples from different sources. Model training primarily uses data samples collected from real-world road cleaning monitoring scenarios from the same source. During collection, both positive sample targets requiring alarms and negative sample targets not requiring alarms need to be considered simultaneously, and images of extremely low quality (unreadable to the human eye) are cleaned out. Both the second and third training sets include positive and negative sample data. The positive sample data includes at least one of the following: scattered household waste sample images, building material pile waste sample images, and household waste pile sample images. The negative sample data includes at least one of the following: non-garbage pile sample images, road pothole sample images, garbage can sample images, garbage bin sample images, truck sample images, garbage truck sample images, road manhole cover sample images, and brick and stone pile sample images. During model training, negative bounding boxes for waste items in scattered household waste sample images, building material piles, and household waste piles are automatically generated. Positive bounding boxes for non-waste items in non-waste pile sample images, road potholes, trash cans, garbage bins, trucks, garbage trucks, road manhole covers, and brick and stone piles are labeled. This enhances the model's resistance to interference from non-waste targets.

[0101] Positive and negative sample construction and sampling: The labeled positive and negative sample images are grouped according to scene similarity. The scene similarity is determined using a perceptual hashing algorithm (phash). Groups with an excessive number of images are downsampled to ensure the balance of data across scenes. Then, the number of bounding boxes for each category in the positive and negative samples in the training data is counted. During model training, positive and negative samples are randomly sampled. The initial sampling ratio is the proportion of the number of bounding boxes for each category in the positive and negative samples to the total number of images, and it is dynamically adjusted based on the training effect.

[0102] Backbone Network: The processed training data is first input into the backbone network for low-level feature extraction. The backbone network consists of four CBS modules, one ELAN, one MP, one ELAN, one MP, one ELAN, one MP, and one ELAN layer connected in series. The CBS module consists of one convolutional layer followed by one batch regularization layer and one SiLu activation function; the ELAN structure is as follows... Figure 3As shown; the MP structure consists of two parallel branches, one of which is a maximum pooling layer connected to a CBS module, and the other is two CBS modules connected in series.

[0103] Neck Network: The main function of this network structure is multi-scale feature fusion to improve the model's ability to identify garbage targets at different scales. The core components are SPPCSPC and ELAN-W. SPPCSPC is an improvement on the feature pyramid structure; ELAN-W adds links to the output connection part of each CBS structure layer on top of the ELAN structure.

[0104] Head Network: This part is the model's output network. Feature maps input from the neck network are fed into three different output branches at different scales. Each output branch consists of a REP followed by a CBM. The REP is a reparameterized convolutional module, consisting of 3x3 convolutions connected in parallel with 1x1 convolution branches during training. If the number of input and output channels and their dimensions are the same, a batch regularization branch is added, and the three branches are summed for the output. During deployment, the parameters of the branches are reparameterized onto the main branch, and the 3x3 main branch convolution is used for output. The CBM consists of convolutions, batch regularization, and a Sigmoid layer. The Sigmoid function is expressed as: f(x) = 1 / (1+e^(-x) / x). -x ).

[0105] Loss Function and Output: During the training phase, the output of the head network is fed into the loss function calculation for backpropagation to optimize the overall network parameters. The loss function consists of three parts: target confidence loss, classification loss, and coordinate regression loss. The target confidence loss uses a logarithmic binary cross-entropy loss, the classification loss uses a logarithmic binary cross-entropy loss plus focal loss, and the coordinate regression loss uses CIoU loss. The specific formulas are as follows:

[0106] focal loss: FL(P)=-α(1-P) γ log(P);

[0107] Where α is a hyperparameter that can suppress the imbalance between positive and negative sample numbers, and γ is a hyperparameter that can control the imbalance between easy and difficult sample numbers.

[0108]

[0109]

[0110]

[0111] Where IoU is the intersection-union ratio of the two bounding boxes, α and v are the aspect ratios, and b represents the parameters for predicting the center coordinates. gt ρ represents the parameter representing the center of the true target bounding box. 2The distance between the center points is represented by , c represents the length of the diagonal of the smallest bounding rectangle of the two rectangles, and w and h represent the width and height of the box.

[0112] This model training employs a multi-level transfer learning technique to address the initial data shortage. It pre-trains the model's feature learning capabilities using massive amounts of data (millions of records) from other scenarios. Then, the backbone and neck network parameters of this pre-trained model are transferred to a general garbage detection scenario for retraining and optimization. The optimized model parameters are further transferred to a mid-to-high-angle video surveillance garbage detection scenario. The neck and head networks are fine-tuned based on a smaller amount of labeled garbage data, while the backbone network parameters are frozen. This approach allows the model to retain the feature extraction capabilities learned from large-scale data while also detecting the required garbage targets in specific scenarios, improving the model's generalization performance and accuracy in low-sample scenarios.

[0113] Please see Figure 4 , Figure 4 This is one of the flowcharts illustrating the waste detection method provided in this embodiment of the invention. For example... Figure 4 As shown in the figure, an embodiment of the present invention provides a waste detection method, which includes the following steps:

[0114] Step 401: Preprocess the acquired target image, which is an image obtained from a video taken from a preset shooting point on the tower. The preprocessing includes scaling and normalization.

[0115] In one example, based on the needs of the environmental protection and transportation industries, cameras suspended from iron towers monitor scenarios requiring road cleaning alerts in real time, including: city streets, highways, expressways, sidewalks, plaza centers, parks and scenic areas, bus stops, lawns and grass, residential areas and stairwells, etc. The cameras capture target images at regular intervals, which are then preprocessed. Preprocessing may involve scaling the target image to 640*640 pixels and normalizing each pixel value to the 0-1 range by dividing by 255, facilitating subsequent feature extraction. High-definition cameras are preferred to acquire high-definition video, which is beneficial for subsequent feature extraction and improves detection accuracy.

[0116] Step 402: Input the target image into the pre-trained third model to generate detection results;

[0117] The processed images are input into a trained garbage object detection model (i.e., the third model), which outputs bounding boxes and confidence information for categories such as scattered household waste, building material piles, and household waste piles. During the training of the third model, the network parameters of the backbone network obtained through transfer learning can be directly utilized. Only the network parameters of the neck and head networks in the third model need to be iterated. Even with a smaller sample size in the third training set, good training results can still be achieved, improving the accuracy of identifying target object categories at preset shooting locations.

[0118] Step 403: Issue an alarm based on the detection results.

[0119] like Figure 5 As shown, after generating the detection results, the garbage detection monitoring alarm information can be post-processed: for the category that needs to be alarmed, it is determined whether the confidence level of its detection result is greater than a preset threshold such as 0.5; if it is, an alarm is output; otherwise, no alarm is issued.

[0120] Then, alarm output feedback is provided: combining camera positioning and three-dimensional (3D) magnification technology improves the accuracy and efficiency of manual judgment. Furthermore, the alarm output feedback results can be used as sample data in a third training set to further enhance the model's training performance.

[0121] This approach employs multi-level transfer learning during model training. First, it pre-trains the first model's feature extraction capabilities using a large batch of training data from a general scenario. Then, it transfers the backbone network parameters from the first model to the backbone network of the second model. Next, it trains the second model's feature recognition capabilities using a medium batch of training data from a specific scenario. Then, it transfers the backbone network parameters from the first model to the backbone network of the third model, and transfers the neck and head network parameters from the second model to the neck and head networks of the third model. Finally, it trains the third model's feature recognition capabilities from a specific perspective using a small batch of training data from a specific perspective within the current scenario. This improves the model's generalization performance, achieving good training results even with limited training data. Consequently, it enhances the accuracy of garbage detection results.

[0122] Please see Figure 6 , Figure 6 This is a schematic diagram of the model training device provided in an embodiment of the present invention. Figure 6 As shown, the model training device 600 provided in this embodiment of the invention includes:

[0123] The first training module 601 is used to train the first model using the first training set so that the accuracy of the first model in detecting target objects reaches a first threshold.

[0124] The first setting module 602 is used to set the network parameters of the backbone network of the second model to the network parameters of the backbone network of the first model.

[0125] The second training module 603 is used to train the second model using the second training set so that the accuracy of the second model in recognizing the target object category reaches the second threshold.

[0126] The second setting module 604 is used to set the network parameters of the backbone network of the third model to the network parameters of the backbone network of the first model, set the network parameters of the neck network of the third model to the network parameters of the neck network of the second model, and set the network parameters of the head network of the third model to the network parameters of the head network of the second model.

[0127] The third training module 605 is used to train the third model using a third training set so that the accuracy of the third model in recognizing the target object category reaches a third threshold.

[0128] Optionally, the model training device 600 also includes:

[0129] The first acquisition module is used to acquire pre-collected garbage sample images;

[0130] The first processing module is used to perform data augmentation processing on the garbage sample images to obtain the second training set.

[0131] Optionally, the model training device 600 also includes:

[0132] The second acquisition module is used to acquire garbage sample images from pre-collected preset shooting locations;

[0133] The second processing module is used to perform data augmentation processing on the garbage sample images at the preset shooting locations to obtain the third training set.

[0134] Optionally, the data enhancement processing includes at least one of splicing processing, chroma, saturation, and luminance HSV processing, offset processing, scaling processing, and flipping processing.

[0135] Optionally, the third training module 605 includes:

[0136] The iterative submodule is used to iterate the network parameters of the neck network and the head network in the third model until the loss function of the output data of the head network in the third model reaches the target threshold. The loss function includes confidence loss, classification loss and coordinate regression loss.

[0137] Optionally, both the second training set and the third training set include positive sample data and negative sample data. The positive sample data includes at least one of scattered household waste sample images, building material pile waste sample images, and household waste pile sample images. The negative sample data includes at least one of non-garbage pile sample images, road pothole sample images, garbage can sample images, garbage bin sample images, truck sample images, garbage truck sample images, road manhole cover sample images, and brick and stone pile sample images.

[0138] The model training device provided in this embodiment of the invention can achieve Figure 1 The various processes implemented in the method embodiments shown are capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0139] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of the waste detection device provided in an embodiment of the present invention. Figure 7 As shown, the waste detection device 700 provided in this embodiment of the invention includes:

[0140] Preprocessing module 701 is used to preprocess the acquired target image, which is an image obtained from a video taken from a preset shooting point on the tower. The preprocessing includes scaling and normalization.

[0141] The generation module 702 is used to input the target image into a pre-trained third model to generate detection results;

[0142] Feedback module 703 is used to provide alarm feedback based on the detection results.

[0143] The waste detection device provided in this embodiment of the invention can achieve Figure 4 The various processes implemented in the method embodiments shown are capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0144] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0146] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A model training method, characterized in that, The method includes: The first model is trained using a first training set so that the accuracy of the first model in detecting target objects reaches a first threshold; wherein, the first model is used to detect target objects in an image; Set the network parameters of the backbone network of the second model to the network parameters of the backbone network of the first model. The second model is trained using a second training set so that the accuracy of the second model in identifying the target object category reaches a second threshold; wherein, the second model is used to identify the category of the target object; The network parameters of the backbone network of the third model are set to the network parameters of the backbone network of the first model, the network parameters of the neck network of the third model are set to the network parameters of the neck network of the second model, and the network parameters of the head network of the third model are set to the network parameters of the head network of the second model. The third model is trained using a third training set so that the accuracy of the third model in identifying the target object category reaches a third threshold; wherein, the third model is used to identify the category of the target object at the preset shooting location; Both the second training set and the third training set include positive sample data and negative sample data. The positive sample data includes at least one of the following: scattered household waste sample images, building material pile waste sample images, and household waste pile sample images. The negative sample data includes at least one of the following: non-garbage pile sample images, road pothole sample images, garbage can sample images, garbage bin sample images, truck sample images, garbage truck sample images, road manhole cover sample images, and brick and stone pile sample images.

2. The method according to claim 1, characterized in that, Before training the second model using the second training set, the method further includes: Acquire images of pre-collected garbage samples; The garbage sample images are subjected to data augmentation processing to obtain the second training set.

3. The method according to claim 1, characterized in that, Before training the third model using the third training set, the method further includes: Acquire pre-collected images of garbage samples at preset shooting locations; Data augmentation processing is performed on the garbage sample images at the preset shooting locations to obtain the third training set.

4. The method according to any one of claims 2 or 3, characterized in that, The data augmentation process includes at least one of the following: splicing, chroma, saturation, and luminance HSV processing, offsetting, scaling, and flipping.

5. The method according to claim 1, characterized in that, The step of training the third model using a third training set to achieve a third threshold in the accuracy of the third model in identifying target object categories includes: The network parameters of the neck network and the head network in the third model are iterated until the loss function of the output data of the head network in the third model reaches the target threshold. The loss function includes confidence loss, classification loss and coordinate regression loss.

6. A waste detection method, characterized in that, The method includes: The target image is preprocessed, which is an image obtained from a video taken from a preset shooting point on the tower. The preprocessing includes scaling and normalization. The target image is input into a third model trained according to any one of claims 1 to 5 to generate detection results; An alarm will be issued based on the detection results.

7. A model training device, characterized in that, The device includes: A first training module is used to train a first model using a first training set so that the accuracy of the first model in detecting target objects reaches a first threshold; wherein, the first model is used to detect target objects in an image; The first setting module is used to set the network parameters of the backbone network of the second model to the network parameters of the backbone network of the first model. The second training module is used to train the second model using the second training set so that the accuracy of the second model in identifying the target object category reaches a second threshold; wherein, the second model is used to identify the category of the target object; The second setting module is used to set the network parameters of the backbone network of the third model to the network parameters of the backbone network of the first model, set the network parameters of the neck network of the third model to the network parameters of the neck network of the second model, and set the network parameters of the head network of the third model to the network parameters of the head network of the second model. The third training module is used to train the third model using a third training set so that the accuracy of the third model in recognizing the target object category reaches a third threshold. The third model is used to identify the category of target objects at preset shooting locations; Both the second training set and the third training set include positive sample data and negative sample data. The positive sample data includes at least one of the following: scattered household waste sample images, building material pile waste sample images, and household waste pile sample images. The negative sample data includes at least one of the following: non-garbage pile sample images, road pothole sample images, garbage can sample images, garbage bin sample images, truck sample images, garbage truck sample images, road manhole cover sample images, and brick and stone pile sample images.

8. The apparatus according to claim 7, characterized in that, The device further includes: The first acquisition module is used to acquire pre-collected garbage sample images; The first processing module is used to perform data augmentation processing on the garbage sample images to obtain the second training set.

9. The apparatus according to claim 7, characterized in that, The device further includes: The second acquisition module is used to acquire garbage sample images from pre-collected preset shooting locations; The second processing module is used to perform data augmentation processing on the garbage sample images at the preset shooting locations to obtain the third training set.

Images