System and method for detecting dangerous objects in garbage can based on deep transfer learning

Abstract

The invention, which relates to the technical field of machine learning, discloses a system and method for detecting a dangerous object in a garbage can based on deep transfer learning, and the system comprises an image collection module, a model construction module, a dangerous object detection module and an alarm module. The image acquisition module is used for acquiring real-time image data in the barrel in real time; the model construction module is used for constructing a training data set through a data enhancement technology on the basis of a neural network model of a transfer learning training basis, and training and updating a trainable layer in the neural network model by using the training data set to obtain a dangerous object detection model; the dangerous object detection module identifies and detects the real-time image data by using a dangerous object detection model to obtain a dangerous object detection result; the alarm module sends out warning information based on the dangerous object detection result. According to the method, the multi-classification capability of the deep learning model can be effectively utilized to accurately distinguish various objects in the garbage can, including hidden dangerous objects, so that the public safety is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a system and method for detecting dangerous objects in trash cans based on deep transfer learning. Background Art

[0002] With the acceleration of urbanization, public safety issues are becoming increasingly serious, especially the hidden dangerous objects in public places and public transportation systems pose a serious threat to people's lives and property. Existing countermeasures mainly include the use of explosion-proof trash cans and transparent trash cans. However, although explosion-proof trash cans can reduce the effects of explosions, they are bulky, expensive, and can only cope with specific threat levels. Although transparent trash cans can reveal the contents of the garbage, they lack aesthetics and are difficult to effectively prevent the placement of dangerous objects. Therefore, there is an urgent need for a more efficient and economical solution to detect dangerous objects hidden in garbage to ensure public safety.

[0003] In recent years, deep learning, especially convolutional neural networks (CNNs), has made remarkable progress in the field of computer vision. CNNs can efficiently identify and classify objects in images with their multi-level feature extraction capabilities. However, training a high-precision CNN model usually requires a large amount of labeled data and high-performance computing resources, which is often difficult to achieve in practical applications. Transfer learning technology can significantly reduce training time and computing resource requirements by utilizing pre-trained CNN models, making model training on small data sets possible. Summary of the invention

[0004] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0005] A dangerous object detection system in a trash can based on deep transfer learning, comprising: an image acquisition module, a model building module, a dangerous object detection module and an alarm module;

[0006] The image acquisition module is used to acquire real-time image data in the bucket in real time;

[0007] The model building module constructs a training data set based on a neural network model trained on the basis of transfer learning through data enhancement technology, and uses the training data set to train and update the trainable layer in the neural network model to obtain a dangerous object detection model;

[0008] The dangerous object detection module uses the dangerous object detection model to identify and detect the real-time image data to obtain a dangerous object detection result;

[0009] The alarm module issues a warning message based on the dangerous object detection result.

[0010] Preferably, the model construction module includes: a data set construction unit, a model training unit and a model updating unit;

[0011] The data set construction unit merges an existing dangerous object image data set and an existing garbage image data set, and uses data enhancement technology to increase the diversity of the merged data set to obtain the training data set;

[0012] The model training unit is used to freeze the early layers of the selected convolutional neural network model, add a number of the trainable layers on the frozen early layers, and train the number of the trainable layers using the training data set to obtain the dangerous object detection model;

[0013] The model updating unit is used to update the dangerous object detection model based on a new dangerous object image and the light environment conditions in the barrel.

[0014] Preferably, the workflow of the data set construction unit includes:

[0015] Collecting the existing dangerous object image dataset and the garbage image dataset, and cleaning and standardizing the dangerous object image dataset;

[0016] and merging the processed dangerous object image dataset and the garbage image dataset to obtain a merged dataset;

[0017] The data enhancement tool library Augmentor is used to perform data enhancement on the merged data set to increase data diversity and scale, thereby obtaining the training data set.

[0018] Preferably, the workflow of the model training unit includes:

[0019] Selecting the pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, wherein the early layers are several convolutional layers at the front of the model;

[0020] Adding a plurality of trainable layers on the frozen early layer, wherein the trainable layers include: a fully connected layer, a batch normalization layer, and an activation function layer;

[0021] The trainable layer learns dangerous object features from the high-level features extracted by the early layer after freezing;

[0022] The dangerous object detection model is obtained by training the plurality of trainable layers after learning the dangerous object features using the training data set.

[0023] Preferably, the training method of the dangerous object detection model includes:

[0024] Initialize the convolutional neural network model using pre-trained weights:

[0025] C initial ←CP pretrained ,

[0026] Among them, C initial Represents the convolutional neural network model, CP pretrained represents the pre-training weights;

[0027] Freeze the earlier layers, add and train the new trainable layers:

[0028]

[0029] Among them, for represents the traversal operation, f represents the fth layer of the model, CP f represents the weight of the fth layer of the model, F frozen represents the set of frozen layers, Represents the set of trainable layers, constant represents the set of frozen layers F frozen The weight remains unchanged, and trainable represents a set of trainable layers The weight of can be updated;

[0030] The convolutional neural network model is preliminarily trained using the cross entropy loss function, where the loss function is:

[0031]

[0032] in, represents the loss function, N represents the total number of samples, yi represents the true label of the i-th sample, represents the predicted probability of the i-th sample;

[0033] Unfreeze some of the frozen early layers, and retrain the convolutional neural network model using a low learning rate to obtain the dangerous object detection model.

[0034] Preferably, the working process of the model updating unit includes:

[0035] Initializing model parameters of the dangerous object detection model using the pre-trained weights;

[0036] The forward propagation calculates the model output value, uses L2 regularization to add a regularization term to the cross entropy loss function, and uses the regularized loss function to calculate the loss, where the regularized loss function is:

[0037]

[0038] in, represents the loss function after regularization, represents the loss function, λ represents the regularization coefficient, CPi represents the model weight, represents the sum of squares of all model weights;

[0039] Compute the gradient via backpropagation:

[0040]

[0041] δ hidden =δ L ·W output ,

[0042] δ input =δ hidden ·W hidden ,

[0043] Among them, δ L represents the gradient of the loss function with respect to the output layer, CP t represents the model parameters at the tth iteration, represents the gradient of the model parameters, represents the loss function, δ hidden represents the gradient of the output layer to the hidden layer, W output represents the weight of the output layer, δ input represents the gradient of the hidden layer to the input layer, W hidden represents the weight of the hidden layer;

[0044] Update the model parameters using the stochastic gradient descent optimizer:

[0045]

[0046] Among them, CP t+1 represents the updated model parameters, and η represents the learning rate.

[0047] The present invention also provides a method for detecting dangerous objects in a trash can based on deep transfer learning. The detection method is applied to any of the above detection systems and comprises the following steps:

[0048] S1. Real-time collection of real-time image data in the bucket;

[0049] S2. Based on the transfer learning training-based neural network model, a training data set is constructed through data enhancement technology, and the training data set is used to train and update the trainable layer in the neural network model to obtain a dangerous object detection model;

[0050] S3. Using the dangerous object detection model to identify and detect the real-time image data, obtain dangerous object detection results;

[0051] S4. Issue a warning message based on the dangerous object detection result.

[0052] Preferably, S2 includes:

[0053] Merging an existing dataset of dangerous objects images and an existing dataset of garbage images, and using data enhancement technology to increase the diversity of the merged dataset to obtain the training dataset;

[0054] Freezing the early layers of the selected convolutional neural network model, adding a number of the trainable layers on the frozen early layers, and training the number of the trainable layers using the training data set to obtain the dangerous object detection model;

[0055] The dangerous object detection model is updated based on the new dangerous object image and the light environment conditions in the barrel.

[0056] Preferably, the method for obtaining the training data set includes:

[0057] Collecting the existing dangerous object image dataset and the garbage image dataset, and cleaning and standardizing the dangerous object image dataset;

[0058] and merging the processed dangerous object image dataset and the garbage image dataset to obtain a merged dataset;

[0059] The data enhancement tool library Augmentor is used to perform data enhancement on the merged data set to increase data diversity and scale, thereby obtaining the training data set.

[0060] Preferably, the method for obtaining the dangerous object detection model includes:

[0061] Selecting the pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, wherein the early layers are several convolutional layers at the front of the model;

[0062] Adding a plurality of trainable layers on the frozen early layer, wherein the trainable layers include: a fully connected layer, a batch normalization layer, and an activation function layer;

[0063] The trainable layer learns dangerous object features from the high-level features extracted by the early layer after freezing;

[0064] The dangerous object detection model is obtained by training the plurality of trainable layers after learning the dangerous object features using the training data set.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention reduces the need for large-scale labeled data and high-performance computing resources by introducing deep transfer learning technology and using pre-trained convolutional neural network models and transfer learning technology, making model training on small data sets possible, significantly improving the accuracy and efficiency of dangerous object detection; through a variety of data enhancement technologies and generative adversarial networks, the training data set is further expanded, the generalization ability and robustness of the model are improved, and real-time detection and classification of various dangerous objects in garbage are achieved; by regularly updating the training data set and model parameters, the system's adaptability to new dangerous objects and environmental changes is ensured. In summary, the present invention provides an efficient and economical solution that effectively improves the safety of public places and public transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0068] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of a basic model of an embodiment of the present invention;

[0070] Figure 3 Schematic diagram of activation function characteristics of an embodiment of the present invention, wherein (a) represents the characteristics of the ReLU activation function, and (b) represents the characteristics of the LReLU and PReLU activation functions. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Embodiment 1

[0074] In this embodiment, if Figure 1 As shown, a dangerous object detection system in a trash can based on deep transfer learning includes: an image acquisition module, a model building module, a dangerous object detection module and an alarm module.

[0075] The image acquisition module is used to collect real-time image data in the bucket. The image acquisition module uses a high-definition camera.

[0076] The model building module is based on a neural network model based on transfer learning training. It constructs a training data set through data enhancement technology, and uses the training data set to train and update the trainable layers in the neural network model to obtain a dangerous object detection model.

[0077] The model building module includes: a dataset building unit, a model training unit and a model updating unit.

[0078] The data set construction unit merges the existing dangerous object image data set and the existing garbage image data set, and uses data enhancement technology to increase the diversity of the merged data set to obtain a training data set.

[0079] The workflow of the dataset construction unit includes: collecting existing dangerous object image datasets and garbage image datasets, and cleaning and standardizing the dangerous object image datasets; merging the processed dangerous object image datasets and garbage image datasets to obtain a merged dataset; using the data enhancement tool library Augmentor to enhance the merged dataset, increase data diversity and scale, and obtain a training dataset.

[0080] In this embodiment, various types of dangerous object images are collected from the Internet, and batch images are downloaded using the Python library image_downloader and browser extensions Imageye and Fatkun. The image format is cleaned and converted, and the collected dangerous object images are merged with the existing garbage image dataset to generate an enhanced dataset DID. Specifically, the Python package Augmentor is used for offline data enhancement, increasing the number of images in the dangerous object training set from 95 to 275. The data enhancement methods applied include rotation (ranging from -5 to 10 degrees), vertical and horizontal flipping, scaling, brightness adjustment, distortion, and shearing, all of which have a 50% probability of application, after which new dangerous object images are generated using generative adversarial networks (GANs).

[0081] The model training unit is used to freeze the early layers of the selected convolutional neural network model, add a number of trainable layers on the frozen early layers, and train the several trainable layers using the training data set to obtain a dangerous object detection model.

[0082] The workflow of the model training unit includes: selecting a pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, which are several convolutional layers at the front of the model; adding several trainable layers on the frozen early layers, which include: fully connected layers, batch normalization layers, and activation function layers; the trainable layers learn dangerous object features from the high-level features extracted from the frozen early layers; using the training data set to train several trainable layers after learning the dangerous object features to obtain a dangerous object detection model.

[0083] In transfer learning, we first select deep neural network models that are pre-trained on large-scale datasets (such as ImageNet), such as VGG16, DenseNet-121, and MobileNetV2. These models significantly improve the object detection effect by learning common features. When migrating to the training dataset, the early layers of the model are usually frozen to retain common low-level features such as edges and textures. On this basis, fully connected layers, batch normalization layers, and activation function layers are added to further learn and adapt to the specific features of the training dataset. During the training process, the weights of these new layers are adjusted through backpropagation, so that the model can effectively detect dangerous objects in the trash can. If there is a new training dataset, the model can continue to train to improve detection accuracy.

[0084] In this embodiment, VGG-16, DenseNet-121, MobileNetV2, NASNetMobile, EfficientNetV2B0, EfficientNetV2L, ConvNeXtSmall and ConvNeXtXLarge are selected as basic models for transfer learning training. Figure 2 As shown. These models have been pre-trained on the ImageNet dataset and have high classification accuracy. In the process of transfer learning, the early layers of the pre-trained model (such as the first several convolutional layers) are selected and the weights of these layers are frozen so that they remain unchanged in subsequent training. These early layers contain rich feature extraction information, which is helpful for learning new tasks. Fully connected layers, batch normalization layers, and activation function layers are added on top of the frozen layers to further learn and adapt to new dataset features. The fully connected layer is used to flatten the feature map of the previous layer and further combine and abstract features through neurons; the batch normalization layer is used to normalize each batch of data, accelerate the training process, and improve the stability and generalization ability of the model; the activation function layer is used to introduce nonlinearity so that the neural network can learn and represent complex functions, thereby improving classification performance. These newly added layers will be responsible for learning more useful information for dangerous object detection tasks from the high-level features extracted by the frozen layers, such as Figure 3 As shown, Figure 3 (a) represents the characteristics of the ReLU activation function. Figure 3 (b) shows the characteristics of LReLU and PReLU activation functions, and f(z) shows the output value corresponding to input z in the neural network model. In this embodiment, after the initial training of the model is completed, if the model performance needs to be improved, some or all layers can be unfrozen and the model can be retrained on the enhanced dataset with a lower learning rate. The performance of the dangerous object detection model on the training dataset is optimized by fine-tuning.

[0085] The training method of the dangerous object detection model includes: initializing the convolutional neural network model using pre-trained weights:

[0086] C initial ←CP pretrained ,

[0087] Among them, C initial Represents the convolutional neural network model, CP pretrained Represent pretrained weights; freeze early layers, add and train new trainable layers:

[0088]

[0089] Among them, for represents the traversal operation, f represents the fth layer of the model, CP f represents the weight of the fth layer of the model, F frozen represents the set of frozen layers, Represents the set of trainable layers, constant represents the set of frozen layers F frozen The weight remains unchanged, and trainable represents a set of trainable layers The weights can be updated; the convolutional neural network model is initially trained using the cross entropy loss function, where the loss function is:

[0090]

[0091] in, represents the loss function, N represents the total number of samples, y i represents the true label of the i-th sample, Represents the predicted probability of the i-th sample; unfreeze some of the frozen early layers, retrain the convolutional neural network model using a low learning rate, and obtain a dangerous object detection model, where the learning rate is:

[0092] η fine-tuning <η initial-training ,

[0093] Among them, η fine-tuning represents the learning rate during fine-tuning, η initial-training Represents the learning rate during initial training.

[0094] The model updating unit is used to update the dangerous object detection model based on the new dangerous object image and the light environment conditions in the barrel.

[0095] In this embodiment, the model update unit collects new dangerous object images and environmental change data in the trash can, including light changes, temperature changes, etc., to enhance the adaptability of the model; retrains the model on the enhanced data set; and updates the model parameters to ensure the accuracy and robustness of the detection.

[0096] The working process of the model update unit includes: initializing the model parameters of the dangerous object detection model using the pre-trained weights:

[0097] CP initial ←CP pretrained ,

[0098] Among them, CP initial Represents the model parameters after initialization, CP pretrained Represents the pre-trained weights; forward propagation calculates the model output value, uses L2 regularization to add a regularization term to the cross entropy loss function, and uses the regularized loss function to calculate the loss, where the regularized loss function is:

[0099]

[0100] in, represents the loss function after regularization, represents the loss function, λ represents the regularization coefficient, CP i represents the model weight, represents the sum of squares of all model weights; the gradient is calculated by backpropagation:

[0101]

[0102] δ hidden =δ L ·W output ,

[0103] δ input =δ hidden ·W hidden ,

[0104] Among them, δ L represents the gradient of the loss function with respect to the output layer, CP t represents the model parameters at the tth iteration, represents the gradient of the model parameters, represents the loss function, δ hidden represents the gradient of the output layer to the hidden layer, W outpuT represents the weight of the output layer, δ input represents the gradient of the hidden layer to the input layer, Whidden represents the weights of the hidden layer; the model parameters are updated using the stochastic gradient descent optimizer:

[0105]

[0106] Among them, CP t+1 Represents the updated model parameters, and η represents the learning rate. The optimization process minimizes the loss function by gradually adjusting the model parameters to improve the performance of the model in the dangerous object detection task.

[0107] The dangerous object detection module uses the dangerous object detection model to identify and detect real-time image data to obtain dangerous object detection results.

[0108] The alarm module issues warning information based on the dangerous object detection results.

[0109] In this embodiment, the overall workflow of the system is as follows: a dangerous object detection module equipped with a convolutional neural network model receives real-time image data from an image acquisition module, and the dangerous object detection module here can select a cloud server; the dangerous object detection module uses a pre-trained convolutional neural network (CNNs) model to analyze and classify images to detect dangerous objects; when a dangerous object is detected, the alarm module triggers an alarm mechanism, including sending alarm information to nearby smartphones, activating local sirens and displaying warning information; sending detailed alarm information to a security monitoring center; managing and monitoring the operating status of the entire detection system, recording and analyzing detection data, and optimizing and adjusting the detection model.

[0110] Embodiment 2

[0111] In this embodiment, in order to verify the effectiveness of the present invention, experiments were conducted and the results were compared with the existing methods.

[0112] 1. Dataset and Experimental Settings

[0113] A new database DID (Dataset for IED Detection) containing dangerous object images is constructed and merged with the existing garbage image dataset to increase the diversity of the dataset through data enhancement technology. The dataset contains various types of dangerous object images and common garbage images. This embodiment uses eight pre-trained convolutional neural network models for transfer learning and trains and tests them on the new dataset.

[0114] 2. Performance evaluation indicators

[0115] The performance of the model is evaluated using indicators such as accuracy, precision, recall, and F1 score. At the same time, the area under the ROC curve (AUC) is calculated to evaluate the classification ability of the model.

[0116] 3. Experimental results

[0117] The models using transfer learning performed well in all indicators, especially the DenseNet-121 model, which achieved the highest values ​​in accuracy, precision, recall, and F1 score.

[0118] 4. Fine-tuning and improvements

[0119] After the initial training, the best performing VGG-16 model was fine-tuned. We further improved the detection accuracy of the model by unfreezing some layers and retraining with a lower learning rate.

[0120] By implementing the above steps and experimental verification, the present invention successfully introduces deep transfer learning technology, which significantly improves the accuracy and efficiency of dangerous object detection. Using multi-classification classifier technology, the present invention realizes real-time and accurate detection and classification of various types of dangerous objects in garbage, providing an efficient and economical safety protection solution for public places and public transportation systems. In addition, by adopting pre-trained convolutional neural network models and transfer learning technology, the present invention effectively reduces the dependence on large-scale labeled data and high-performance computing resources, making model training on small-scale data sets possible. Specifically, VGG16, DenseNet-121, MobileNetV2, etc. are selected as basic models for deep transfer learning. These models can achieve efficient image classification under limited resource conditions. By combining a variety of data enhancement techniques and generative adversarial networks (GANs), the training data set is further enriched, and the generalization ability and robustness of the model are significantly enhanced. The multi-classification classifier used can handle classification tasks of multiple categories at the same time, not only identifying dangerous objects, but also distinguishing other non-dangerous objects (such as ordinary garbage). Finally, by regularly updating the training data set and model parameters, the present invention ensures the rapid adaptation and continuous optimization of the system to new dangerous objects and environmental changes.

[0121] Embodiment 3

[0122] In this embodiment, a method for detecting dangerous objects in a trash can based on deep transfer learning includes the following steps:

[0123] S1. Collect real-time image data in the bucket in real time.

[0124] S2. Based on the transfer learning training-based neural network model, a training data set is constructed through data enhancement technology, and the training data set is used to train and update the trainable layers in the neural network model to obtain a dangerous object detection model.

[0125] S2 includes:

[0126] S2.1. Merge the existing dangerous object image dataset and the existing garbage image dataset, and use data enhancement technology to increase the diversity of the merged dataset to obtain a training dataset.

[0127] The method for obtaining a training data set includes: collecting existing dangerous object image data sets and garbage image data sets, and cleaning and standardizing the dangerous object image data sets; merging the processed dangerous object image data sets and garbage image data sets to obtain a merged data set; using the data enhancement tool library Augmentor to enhance the merged data set to increase data diversity and scale to obtain a training data set.

[0128] S2.2. Freeze the early layers of the selected convolutional neural network model, add several trainable layers on the frozen early layers, and train the several trainable layers using the training data set to obtain a dangerous object detection model.

[0129] The method for obtaining a dangerous object detection model includes: selecting a pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, the early layers being several convolutional layers at the front of the model; adding several trainable layers on the frozen early layers, the trainable layers including: a fully connected layer, a batch normalization layer and an activation function layer; the trainable layers learn dangerous object features from high-level features extracted from the frozen early layers; using a training data set to train several trainable layers after learning the dangerous object features, to obtain a dangerous object detection model.

[0130] The training method of the dangerous object detection model includes: initializing the convolutional neural network model using pre-trained weights:

[0131] C initial ←CP pretrained ,

[0132] Among them, C initial Represents the convolutional neural network model, CP pretraiined Represent pretrained weights; freeze early layers, add and train new trainable layers:

[0133]

[0134] Among them, for represents the traversal operation, f represents the fth layer of the model, CP f represents the weight of the fth layer of the model, F frozenrepresents the set of frozen layers, Represents the set of trainable layers, constant represents the set of frozen layers F frozen The weight remains unchanged, and trainable represents a set of trainable layers The weights can be updated; the convolutional neural network model is initially trained using the cross entropy loss function, where the loss function is:

[0135]

[0136] in, represents the loss function, N represents the total number of samples, yi represents the true label of the i-th sample, Represents the predicted probability of the i-th sample; unfreeze some of the frozen early layers, retrain the convolutional neural network model using a low learning rate, and obtain a dangerous object detection model.

[0137] S2.3. Update the hazardous object detection model based on the new hazardous object image and the light environment conditions inside the barrel.

[0138] The method for updating the dangerous object detection model includes: initializing the model parameters of the dangerous object detection model using pre-trained weights; forward propagating to calculate the model output value, adding a regularization term to the cross entropy loss function using L2 regularization, and calculating the loss using the regularized loss function, wherein the regularized loss function is:

[0139]

[0140] in, represents the loss function after regularization, represents the loss function, λ represents the regularization coefficient, CP i represents the model weight, represents the sum of squares of all model weights; the gradient is calculated by backpropagation:

[0141]

[0142] δ hidden =δ L ·W output ,

[0143] δ input =δ hidden ·W hidden ,

[0144] Among them, δ L represents the gradient of the loss function with respect to the output layer, CP t represents the model parameters at the tth iteration, represents the gradient of the model parameters, represents the loss function, δhidden represents the gradient of the output layer to the hidden layer, W output represents the weight of the output layer, δ input represents the gradient of the hidden layer to the input layer, W hidden represents the weights of the hidden layer; the model parameters are updated using the stochastic gradient descent optimizer:

[0145]

[0146] Among them, CP t+1 represents the updated model parameters, and η represents the learning rate.

[0147] S3. Use the dangerous object detection model to identify and detect the real-time image data to obtain the dangerous object detection result.

[0148] S4. Issue a warning message based on the dangerous object detection results.

[0149] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A dangerous object detection system in a trash can based on deep transfer learning, characterized in that: include: Image acquisition module, model building module, dangerous object detection module and alarm module; The image acquisition module is used to acquire real-time image data in the bucket in real time; The model building module constructs a training data set based on a neural network model trained on the basis of transfer learning through data enhancement technology, and uses the training data set to train and update the trainable layer in the neural network model to obtain a dangerous object detection model; The dangerous object detection module uses the dangerous object detection model to identify and detect the real-time image data to obtain a dangerous object detection result; The alarm module issues a warning message based on the dangerous object detection result.

2. According to claim 1, a dangerous object detection system in a trash can based on deep transfer learning is characterized in that: The model building module includes: a data set building unit, a model training unit and a model updating unit; The data set construction unit merges an existing dangerous object image data set and an existing garbage image data set, and uses data enhancement technology to increase the diversity of the merged data set to obtain the training data set; The model training unit is used to freeze the early layers of the selected convolutional neural network model, add a number of the trainable layers on the frozen early layers, and train the number of the trainable layers using the training data set to obtain the dangerous object detection model; The model updating unit is used to update the dangerous object detection model based on a new dangerous object image and the light environment conditions in the barrel.

3. According to claim 2, a dangerous object detection system in a trash can based on deep transfer learning is characterized in that: The workflow of the dataset construction unit includes: Collecting the existing dangerous object image dataset and the garbage image dataset, and cleaning and standardizing the dangerous object image dataset; and merging the processed dangerous object image dataset and the garbage image dataset to obtain a merged dataset; The data enhancement tool library Augmentor is used to perform data enhancement on the merged data set to increase data diversity and scale, thereby obtaining the training data set.

4. According to claim 2, a dangerous object detection system in a trash can based on deep transfer learning is characterized in that: The workflow of the model training unit includes: Selecting the pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, wherein the early layers are several convolutional layers at the front of the model; Adding a plurality of trainable layers on the frozen early layer, wherein the trainable layers include: a fully connected layer, a batch normalization layer, and an activation function layer; The trainable layer learns dangerous object features from the high-level features extracted by the early layer after freezing; The dangerous object detection model is obtained by training the plurality of trainable layers after learning the dangerous object features using the training data set.

5. According to claim 4, a dangerous object detection system in a trash can based on deep transfer learning is characterized in that: The training method of the dangerous object detection model includes: Initialize the convolutional neural network model using pre-trained weights: C initial ←CP pretraomed , Among them, C initial Represents the convolutional neural network model, CP pretrained represents the pre-training weights; Freeze the earlier layers, add and train the new trainable layers: Among them, for represents the traversal operation, f represents the fth layer of the model, CP f represents the weight of the fth layer of the model, F frozen represents the set of frozen layers, Represents the set of trainable layers, constant represents the set of frozen layers F frozen The weight remains unchanged, and trainable represents a set of trainable layers The weight of can be updated; The convolutional neural network model is preliminarily trained using the cross entropy loss function, where the loss function is: in, represents the loss function, N represents the total number of samples, y i represents the true label of the i-th sample, represents the predicted probability of the i-th sample; Unfreeze some of the frozen early layers, and retrain the convolutional neural network model using a low learning rate to obtain the dangerous object detection model.

6. According to claim 5, a dangerous object detection system in a trash can based on deep transfer learning is characterized in that: The working process of the model updating unit includes: Initializing model parameters of the dangerous object detection model using the pre-trained weights; The forward propagation calculates the model output value, uses L2 regularization to add a regularization term to the cross entropy loss function, and uses the regularized loss function to calculate the loss, where the regularized loss function is: in, represents the loss function after regularization, represents the loss function, λ represents the regularization coefficient, CP i represents the model weight, represents the sum of squares of all model weights; Compute the gradient via backpropagation: d hidden =d L ·W output , d input =d hidden ·W hidden , Among them, δ L represents the gradient of the loss function with respect to the output layer, CP t represents the model parameters at the tth iteration, represents the gradient of the model parameters, represents the loss function, δ hidden represents the gradient of the output layer to the hidden layer, W output represents the weight of the output layer, δ input represents the gradient of the hidden layer to the input layer, W hidden represents the weight of the hidden layer; Update the model parameters using the stochastic gradient descent optimizer: Among them, CP t+1 represents the updated model parameters, and η represents the learning rate.

7. A method for detecting dangerous objects in a trash can based on deep transfer learning, the method being applied to the detection system according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Real-time collection of real-time image data in the bucket; S2. Based on the transfer learning training-based neural network model, a training data set is constructed through data enhancement technology, and the training data set is used to train and update the trainable layer in the neural network model to obtain a dangerous object detection model; S3. Using the dangerous object detection model to identify and detect the real-time image data, obtain dangerous object detection results; S4. Issue a warning message based on the dangerous object detection result.

8. According to claim 7, a method for detecting dangerous objects in a trash can based on deep transfer learning is characterized in that: The S2 includes: Merging an existing dataset of dangerous objects images and an existing dataset of garbage images, and using data enhancement technology to increase the diversity of the merged dataset to obtain the training dataset; Freezing the early layers of the selected convolutional neural network model, adding a number of the trainable layers on the frozen early layers, and training the number of the trainable layers using the training data set to obtain the dangerous object detection model; The dangerous object detection model is updated based on the new dangerous object image and the light environment conditions in the barrel.

9. The method for detecting dangerous objects in trash cans based on deep transfer learning according to claim 8, characterized in that: The method for obtaining the training data set includes: Collecting the existing dangerous object image dataset and the garbage image dataset, and cleaning and standardizing the dangerous object image dataset; and merging the processed dangerous object image dataset and the garbage image dataset to obtain a merged dataset; The data enhancement tool library Augmentor is used to perform data enhancement on the merged data set to increase data diversity and scale, thereby obtaining the training data set.

10. The method for detecting dangerous objects in trash cans based on deep transfer learning according to claim 8, characterized in that: The method for obtaining the dangerous object detection model includes: Selecting the pre-trained convolutional neural network model through transfer learning, freezing the early layers of the convolutional neural network model, wherein the early layers are several convolutional layers at the front of the model; Adding a plurality of trainable layers on the frozen early layer, wherein the trainable layers include: a fully connected layer, a batch normalization layer, and an activation function layer; The trainable layer learns dangerous object features from the high-level features extracted by the early layer after freezing; The dangerous object detection model is obtained by training the plurality of trainable layers after learning the dangerous object features using the training data set.

Images