Residue soil vehicle supervision method and system based on deep learning

Abstract

The invention discloses a slag car supervision method and system based on deep learning, and belongs to the technical field of slag car supervision, the slag car supervision method and system based on deep learning comprises the following specific steps: step 1, constructing a slag car data set, and collecting data through monitoring of a construction site entrance and exit, the method comprises the following steps of: performing frame extraction, duplicate removal and other operations on monitored video stream data to obtain a slag car data set picture, labeling license plate information of a slag car to obtain a training data set of an OCR model, and constructing a target detection network model based on deep learning; the image recognition model and the character recognition model are constructed by adopting a deep learning method, the slag car, the loading of the slag car, the appearance of the slag car, the license plate of the slag car and the characters of the license plate are recognized through images, and when an event occurs, the system can automatically record and report, so that manpower and material resources are greatly reduced, and the working efficiency is improved. And the intelligence and the working efficiency of the system are improved.

Description

Technical Field

[0001] The present invention belongs to the field of slag truck supervision technology, and specifically relates to a slag truck supervision method and system based on deep learning. Background Art

[0002] With the improvement of the level of refined urban management, effective supervision of construction site muck trucks is an important part of urban construction. If muck trucks are overloaded, transported without a license, or leak when entering or leaving the construction site, it will not only seriously pollute the urban environment and affect the city's image, but also endanger road safety. Supervision of construction site muck trucks can effectively detect and stop these illegal behaviors at the source, reduce dust pollution and road pollution, and protect the city's ecological environment.

[0003] Traditional manual supervision methods have many shortcomings, such as slow response speed and difficulty in law enforcement. The supervision of muck trucks involves multiple departments, such as urban management, public security, and transportation. Manual supervision is difficult to achieve efficient cross-departmental coordination, which brings huge challenges to urban management. Therefore, a more efficient and intelligent supervision method is needed.

[0004] The origin of deep learning can be traced back to the 1940s, when people began to study the computational model of neurons. However, due to the limitations of computing power and data volume at the time, progress was slow. With the rapid development of computer technology, especially in the late 20th century and early 21st century, the explosive growth of data volume and the emergence of graphics processing units (GPUs) have provided a foundation for the vigorous development of deep learning. Therefore, a method and system for monitoring muck trucks based on deep learning are urgently needed. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method and system for monitoring a muck truck based on deep learning.

[0006] The technical solution adopted to solve the above technical problems is: a method for supervising muck trucks based on deep learning, including the following specific steps:

[0007] Step 1: Build a muck truck dataset. Collect data through monitoring at the entrance and exit of the construction site. Perform frame extraction and deduplication operations on the monitored video stream data to obtain the muck truck dataset image. Label the muck truck license plate information to obtain the training dataset of the OCR model. Build a target detection network model based on deep learning.

[0008] Step 2: Use the muck truck dataset to train the muck truck recognition model. After the model recognizes the muck truck, it obtains the confidence and position information of the target, and cuts out the muck truck target image based on its position information. Use the muck truck license plate dataset to train the muck truck license plate recognition model, pass in the muck truck target image for license plate position recognition, obtain the license plate position information, cut out the license plate image, and form a muck truck license plate text dataset. Use the muck truck loading dataset to train the muck truck loading recognition model, and recognize the muck truck image. Use the muck truck appearance dataset to train the muck truck vehicle appearance recognition model, pass in the muck truck image to recognize the appearance of the muck truck, and judge whether the muck truck is clean. Build a text recognition model based on deep learning, pass the license plate text dataset into the model for training, and obtain a license plate text recognition model based on deep learning;

[0009] Step 3: Build an IoT system for muck truck identification;

[0010] Step 4: Define the exit area and identify the muck truck through monitoring equipment;

[0011] Step 5: If it is not a muck truck, it will be directly released. If it is a muck truck, the muck truck loading will be identified. If it is not fully loaded, it will be reloaded. If it is fully loaded, it will enter the next stage.

[0012] Step 6: Identify the appearance of the dump truck. If the appearance is not clean, guide it into the washing area until the appearance is clean. Then identify the license plate of the dump truck. After the identification is completed, report to the system and allow the dump truck to leave the road.

[0013] Through the above technical solution, a dump truck supervision system was built, and the deep learning method was used to construct the image recognition model and the text recognition model. Through image recognition of dump trucks, dump truck loading, dump truck appearance, dump truck license plates and license plate text, when an incident occurs, the system can automatically record and report it, which greatly reduces manpower and material resources and improves the intelligence and work efficiency of the system.

[0014] Furthermore, it includes four major subsystems: data collection module, data preprocessing module, model building module and Internet of Things system module. The data collection module is responsible for collecting the appearance, cargo condition and license plate information data of the dump truck through monitoring at the entrance and exit of the construction site. The data preprocessing module is responsible for extracting frames, deduplicating and labeling the data collected by the data collection module.

[0015] Furthermore, the model building module is responsible for building a target detection network model based on deep learning. It is divided into three parts. First, a convolution layer is established, including convolution, BN (Batch Normalization) layer, and activation layer. The input image is subjected to 3×3 convolution to extract features.

[0016] Secondly, the middle part mainly extracts the features of the image in depth, which includes multiple convolution blocks. The convolution block consists of convolution, attention set, and connection operations. The input channel is converted into an expansion channel through 1×1 convolution, and the 3×3 convolution is used as a new expansion channel to connect to the upper layer. The expansion channel is converted into an output channel through 1×1 convolution.

[0017] The attention mechanism is used to enhance the feature expression capability without increasing the computational cost. The number of channels, height, and width of the input feature map are processed. Global average pooling along the width is performed on each channel to capture its information in the height direction. At the same time, global average pooling along the height is performed on each channel to capture its information in the width direction. The input features are aggregated along the height and width directions respectively to generate two feature maps containing direction-specific information. The feature maps are batch normalized, and then the sigmoid activation function is used to generate two attention maps representing the attention weights in the width and width directions respectively. The weights are multiplied with the original feature map to enhance the attention of the input feature map in the width and height directions, strengthen important features, and weaken minor features.

[0018] At the same time, residual connections are introduced to ensure that the input and output of the convolutional block are the same. Residual connections can alleviate the gradient vanishing problem in the network and accelerate the training process of the model by adding the input feature map to the output feature map.

[0019] Finally, the output part is constructed. The global average pooling operation downsamples the feature map to reduce the dimension of the feature and extract the global features. The 1×1 convolution layer converts the output of the global average pooling into the final category prediction. The softmax activation function is then used to normalize the category prediction to obtain the distribution probability and location information of each class.

[0020] Through the above technical solution, a dump truck recognition model based on deep learning was built, and an attention mechanism was added to the network structure, which can dynamically adjust the focus so that the model can increase the weight of important features and reduce background interference during the calculation process. At the same time, it can also reduce the calculation of parameters and improve the efficiency of model recognition.

[0021] Furthermore, the model building module is also responsible for building a text recognition model based on deep learning, which is divided into feature extraction layer, feature fusion layer, and output layer;

[0022] Since there are three types of license plates, namely normal license plates, license plates placed on the roof and license plates printed on the body, and the sizes of license plate text data are different, the feature pyramid network is first used to gradually reduce the spatial dimension of the feature map through continuous convolution and pooling operations, while increasing the number of channels to extract high-level semantic information. On the other hand, the resolution of deep feature maps is enhanced through a top-down path. The main operation is to perform an up-sampling operation on the high-level feature map, which has a similar spatial resolution to the low-level feature map. A 1×1 convolution is introduced to combine the high-resolution information of the low-level feature layer with the semantic information of the high-level feature map, and then match the channel input of the feature map. The feature map of the top-down path and the feature map of the bottom-up path are fused to detect texts of different sizes.

[0023] Among them, the large feature layer detects small texts, and the small feature layer detects large texts. Then, the feature fusion layer extracts the last layer of features and sends them to the pooling layer, amplifies the features by 1 times, connects the amplified features with the feature map of the previous layer, and performs convolution operations with kernel sizes of 1×1 and 3×3 on the connected feature layer. The results are output to the output layer through 32-kernel, 3×3 convolutions;

[0024] Finally, the output layer outputs text information and confidence information of the detection box, and the license plate text dataset is passed into the model for training to obtain a license plate text recognition model based on deep learning.

[0025] Through the above technical solution, a text recognition model based on deep learning is built. Since there are three types of license plates, namely normal license plates, license plates placed on the roof and license plates printed on the body, license plate texts of different sizes will affect recognition. Therefore, a feature pyramid network is introduced to detect license plates of different sizes. By successively halving the convolution layer and successively doubling the convolution kernel, feature maps of different sizes are extracted to achieve recognition of license plate texts of different sizes. Among them, the large feature layer detects small license plate texts, and the small feature layer detects large license plate texts, thereby improving the accuracy of license plate text recognition for muck trucks.

[0026] Furthermore, the data preprocessing module includes constructing a muck truck dataset, collecting data through monitoring at the entrance and exit of the construction site, extracting frames and removing duplicates from the monitored video stream data to obtain a muck truck dataset image, and annotating the muck truck license plate information to obtain a training dataset for the OCR model;

[0027] The labelimg annotation tool is used to annotate the dump trucks in the dataset. The labels are dump truck (dregs), dump truck empty (empty), dump truck full (full), dirty dump truck (dirty), clean dump truck (clean), and dump truck license plate (plate). Each image with a labeled target has a corresponding xml label file corresponding to it.

[0028] Among them, the dump truck label is the data set of the dump truck recognition model, the dump truck compartment empty and dump truck compartment fully loaded labels are the data set of the dump truck loading recognition model, the dirty dump truck and clean dump truck labels are the data set of the dump truck appearance recognition model, and the dump truck license plate label is the label of the dump truck license plate recognition model.

[0029] Through the above technical solution, the data processing speed can be improved by converting various information of the muck truck into corresponding labels.

[0030] Furthermore, the IoT system module includes monitoring equipment, edge smart box, display, mouse and keyboard, and communication equipment;

[0031] The monitoring equipment is connected to the edge intelligent box for real-time monitoring. After the monitoring can be displayed normally, the deep learning model can be deployed. After the deployment is completed, the recognition status can be checked through the edge intelligent gateway system. Muck truck recognition, license plate recognition and license plate text recognition, muck truck loading recognition, and muck truck vehicle appearance recognition can all be pushed and viewed in real time. If the license plate is not entered into the system, manual intervention is required at the entrance of the construction site.

[0032] The beneficial effects of the present invention are as follows: the present invention constructs a dump truck supervision system based on deep learning, installs monitoring equipment at the entrance and exit of the smart construction site, and demarcates the entry and exit areas. When the dump truck enters the identification area, the system starts working. First, the dump truck is identified on the monitoring screen to determine whether it is a dump truck. If not, it is determined that other vehicles can enter and exit the construction site. If it is a dump truck, the loading of the dump truck is identified. The loading identification of the dump truck can effectively count the full load of the dump truck. If it is not fully loaded, it returns for reloading. If it is fully loaded, the appearance of the dump truck is identified. The dump truck will be washed once when leaving the construction site. The appearance of the dump truck is identified in the area to be exited, and mud can be found in the body, wheels and other places of the dump truck in time. This identification can effectively avoid the pollution of the environment by the dump truck when it is driving on the road. When the system determines that the dump truck is dirty, it returns to the washing area for washing. When the dump truck is clean, the license plate text recognition is performed, the license plate is recorded and reported to the edge intelligent gateway system, and the dump truck is allowed to exit. The system is deployed to edge devices, and the dump truck event reports can be viewed in the edge intelligent gateway system for event management. The system can intelligently, comprehensively and efficiently monitor the conditions of dump trucks entering and leaving the construction site, greatly reducing manpower and improving the efficiency of supervision.

[0033] When building a target detection model based on deep learning, adding an attention mechanism to the network structure can dynamically adjust the focus, so that the model can increase the weight of important features during the calculation process and reduce background interference. At the same time, it can also reduce the calculation of parameters and improve the efficiency of model recognition. When building a text recognition model based on deep learning, a feature pyramid network is introduced to detect license plates of different sizes. By successively halving the convolution layer and doubling the convolution kernel, feature maps of different sizes are extracted to achieve recognition of license plate text of different sizes. Among them, the large feature layer detects small license plate text, and the small feature layer detects large license plate text, which improves the accuracy of muck truck license plate text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a regulatory flow chart of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] like Figure 1 As shown, a method and system for monitoring a muck truck based on deep learning in this embodiment include the following specific steps:

[0037] Step 1: Build a muck truck dataset. Collect data through monitoring at the entrance and exit of the construction site. Perform frame extraction and deduplication operations on the monitored video stream data to obtain the muck truck dataset image. Label the muck truck license plate information to obtain the training dataset of the OCR model. Build a target detection network model based on deep learning.

[0038] Step 2: Use the muck truck dataset to train the muck truck recognition model. After the model recognizes the muck truck, it obtains the confidence and position information of the target, and cuts out the muck truck target image based on its position information. Use the muck truck license plate dataset to train the muck truck license plate recognition model, pass in the muck truck target image for license plate position recognition, obtain the license plate position information, cut out the license plate image, and form a muck truck license plate text dataset. Use the muck truck loading dataset to train the muck truck loading recognition model, and recognize the muck truck image. Use the muck truck appearance dataset to train the muck truck vehicle appearance recognition model, pass in the muck truck image to recognize the appearance of the muck truck, and judge whether the muck truck is clean. Build a text recognition model based on deep learning, pass the license plate text dataset into the model for training, and obtain a license plate text recognition model based on deep learning;

[0039] Step 3: Build an IoT system for muck truck identification;

[0040] Step 4: Define the exit area and identify the muck truck through monitoring equipment;

[0041] Step 5: If it is not a muck truck, it will be directly released. If it is a muck truck, the muck truck loading will be identified. If it is not fully loaded, it will be reloaded. If it is fully loaded, it will enter the next stage.

[0042] Step 6: Identify the appearance of the dump truck. If the appearance is not clean, guide it into the washing area until the appearance is clean. Then identify the license plate of the dump truck. After the identification is completed, report to the system and allow the dump truck to leave the road.

[0043] It includes four subsystems: data collection module, data preprocessing module, model building module and Internet of Things system module. The data collection module is responsible for collecting the appearance, cargo loading and license plate information of the muck truck through the monitoring of the construction site entrance and exit. The data preprocessing module is responsible for extracting frames, de-duplicating and labeling the data collected by the data collection module. The model building module is responsible for building a target detection network model based on deep learning, which is divided into three parts. First, a convolution layer is established, including convolution, BN (Batch Normalization) layer and activation layer. The input image is extracted through 3×3 convolution.

[0044] Secondly, the middle part mainly extracts the features of the image in depth, which includes multiple convolution blocks. The convolution block consists of convolution, attention set, and connection operations. The input channel is converted into an expansion channel through 1×1 convolution, and the 3×3 convolution is used as a new expansion channel to connect to the upper layer. The expansion channel is converted into an output channel through 1×1 convolution.

[0045] The attention mechanism is used to enhance the feature expression capability without increasing the computational cost. The number of channels, height, and width of the input feature map are processed. Global average pooling along the width is performed on each channel to capture its information in the height direction. At the same time, global average pooling along the height is performed on each channel to capture its information in the width direction. The input features are aggregated along the height and width directions respectively to generate two feature maps containing direction-specific information. The feature maps are batch normalized, and then the sigmoid activation function is used to generate two attention maps representing the attention weights in the width and width directions respectively. The weights are multiplied with the original feature map to enhance the attention of the input feature map in the width and height directions, strengthen important features, and weaken minor features.

[0046] At the same time, residual connections are introduced to ensure that the input and output of the convolutional block are the same. Residual connections can alleviate the gradient vanishing problem in the network and accelerate the training process of the model by adding the input feature map to the output feature map.

[0047] Finally, the output part is constructed. The global average pooling operation downsamples the feature map to reduce the dimension of the feature and extract the global features. The 1×1 convolution layer converts the output of the global average pooling into the final category prediction. The softmax activation function is then used to normalize the category prediction to obtain the distribution probability and location information of each class.

[0048] The model building module is also responsible for building a text recognition model based on deep learning, which is divided into feature extraction layer, feature fusion layer, and output layer;

[0049] Since there are three types of license plates, namely normal license plates, license plates placed on the roof and license plates printed on the body, and the sizes of license plate text data are different, the feature pyramid network is first used to gradually reduce the spatial dimension of the feature map through continuous convolution and pooling operations, while increasing the number of channels to extract high-level semantic information. On the other hand, the resolution of deep feature maps is enhanced through a top-down path. The main operation is to perform an up-sampling operation on the high-level feature map, which has a similar spatial resolution to the low-level feature map. A 1×1 convolution is introduced to combine the high-resolution information of the low-level feature layer with the semantic information of the high-level feature map, and then match the channel input of the feature map. The feature map of the top-down path and the feature map of the bottom-up path are fused to detect texts of different sizes.

[0050] Among them, the large feature layer detects small texts, and the small feature layer detects large texts. Then, the feature fusion layer extracts the last layer of features and sends them to the pooling layer, amplifies the features by 1 times, connects the amplified features with the feature map of the previous layer, and performs convolution operations with kernel sizes of 1×1 and 3×3 on the connected feature layer. The results are output to the output layer through 32-kernel, 3×3 convolutions;

[0051] Finally, the output layer outputs text information and confidence information of the detection box, and the license plate text dataset is passed into the model for training to obtain a license plate text recognition model based on deep learning.

[0052] The data preprocessing module includes constructing a muck truck dataset, collecting data through monitoring of the construction site entrance and exit, extracting frames and removing duplicates from the monitored video stream data, obtaining a muck truck dataset image, and annotating the muck truck license plate information to obtain an OCR model training dataset;

[0053] The labelimg annotation tool is used to annotate the dump trucks in the dataset. The labels are dump truck (dregs), dump truck empty (empty), dump truck full (full), dirty dump truck (dirty), clean dump truck (clean), and dump truck license plate (plate). Each image with a labeled target has a corresponding xml label file corresponding to it.

[0054] Among them, the dump truck label is the data set of the dump truck recognition model, the dump truck compartment empty and dump truck compartment fully loaded labels are the data set of the dump truck loading recognition model, the dirty dump truck and clean dump truck labels are the data set of the dump truck appearance recognition model, and the dump truck license plate label is the label of the dump truck license plate recognition model.

[0055] The IoT system module includes monitoring equipment, edge smart box, display, mouse and keyboard, and communication equipment;

[0056] The monitoring equipment is connected to the edge intelligent box for real-time monitoring. After the monitoring can be displayed normally, the deep learning model can be deployed. After the deployment is completed, the recognition status can be checked through the edge intelligent gateway system. Muck truck recognition, license plate recognition and license plate text recognition, muck truck loading recognition, and muck truck vehicle appearance recognition can all be pushed and viewed in real time. If the license plate is not entered into the system, manual intervention is required at the entrance of the construction site.

[0057] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A method for supervising muck trucks based on deep learning, characterized in that: The specific steps include: Step 1: Build a muck truck dataset. Collect data through monitoring at the entrance and exit of the construction site. Perform frame extraction and deduplication operations on the monitored video stream data to obtain the muck truck dataset image. Label the muck truck license plate information to obtain the training dataset of the OCR model. Build a target detection network model based on deep learning. Step 2: Use the muck truck dataset to train the muck truck recognition model. After the model recognizes the muck truck, it obtains the confidence and position information of the target, and cuts out the muck truck target image based on its position information. Use the muck truck license plate dataset to train the muck truck license plate recognition model, pass in the muck truck target image for license plate position recognition, obtain the license plate position information, cut out the license plate image, and form a muck truck license plate text dataset. Use the muck truck loading dataset to train the muck truck loading recognition model, and recognize the muck truck image. Use the muck truck appearance dataset to train the muck truck vehicle appearance recognition model, pass in the muck truck image to recognize the appearance of the muck truck, and judge whether the muck truck is clean. Build a text recognition model based on deep learning, pass the license plate text dataset into the model for training, and obtain a license plate text recognition model based on deep learning; Step 3: Build an IoT system for muck truck identification; Step 4: Define the exit area and identify the muck truck through monitoring equipment; Step 5: If it is not a muck truck, it will be directly released. If it is a muck truck, the muck truck loading will be identified. If it is not fully loaded, it will be reloaded. If it is fully loaded, it will enter the next stage. Step 6: Identify the appearance of the dump truck. If the appearance is not clean, guide it into the washing area until the appearance is clean. Then identify the license plate of the dump truck. After the identification is completed, report to the system and allow the dump truck to leave the road.

2. A deep learning-based muck truck monitoring system according to claim 1, characterized in that: It includes four subsystems: data collection module, data preprocessing module, model building module and Internet of Things system module. The data collection module is responsible for collecting the appearance, cargo loading and license plate information data of the dump truck through monitoring at the entrance and exit of the construction site. The data preprocessing module is responsible for extracting frames, deduplicating and labeling the data collected by the data collection module.

3. A deep learning-based muck truck monitoring system according to claim 2, characterized in that: The model building module is responsible for building a target detection network model based on deep learning. It is divided into three parts. First, a convolution layer is established, including convolution, BN (Batch Normalization) layer, and activation layer. The input image is subjected to 3×3 convolution to extract features. Secondly, the middle part mainly extracts the features of the image in depth, which includes multiple convolution blocks. The convolution block consists of convolution, attention set, and connection operations. The input channel is converted into an expansion channel through 1×1 convolution, and the 3×3 convolution is used as a new expansion channel to connect to the upper layer. The expansion channel is converted into an output channel through 1×1 convolution. The attention mechanism is used to enhance the feature expression capability without increasing the computational cost. The number of channels, height, and width of the input feature map are processed. Global average pooling along the width is performed on each channel to capture its information in the height direction. At the same time, global average pooling along the height is performed on each channel to capture its information in the width direction. The input features are aggregated along the height and width directions respectively to generate two feature maps containing direction-specific information. The feature maps are batch normalized, and then the sigmoid activation function is used to generate two attention maps representing the attention weights in the width and width directions respectively. The weights are multiplied with the original feature map to enhance the attention of the input feature map in the width and height directions, strengthen important features, and weaken minor features. At the same time, residual connections are introduced to ensure that the input and output of the convolutional block are the same. Residual connections can alleviate the gradient vanishing problem in the network and accelerate the training process of the model by adding the input feature map to the output feature map. Finally, the output part is constructed. The global average pooling operation downsamples the feature map to reduce the dimension of the feature and extract the global features. The 1×1 convolution layer converts the output of the global average pooling into the final category prediction. The softmax activation function is then used to normalize the category prediction to obtain the distribution probability and location information of each class.

4. A deep learning-based muck truck monitoring system according to claim 3, characterized in that: The model building module is also responsible for building a text recognition model based on deep learning, which is divided into feature extraction layer, feature fusion layer, and output layer; Since there are three types of license plates, namely normal license plates, license plates placed on the roof and license plates printed on the body, and the sizes of license plate text data are different, the feature pyramid network is first used to gradually reduce the spatial dimension of the feature map through continuous convolution and pooling operations, while increasing the number of channels to extract high-level semantic information. On the other hand, the resolution of deep feature maps is enhanced through a top-down path. The main operation is to perform an up-sampling operation on the high-level feature map, which has a similar spatial resolution to the low-level feature map. A 1×1 convolution is introduced to combine the high-resolution information of the low-level feature layer with the semantic information of the high-level feature map, and then match the channel input of the feature map. The feature map of the top-down path and the feature map of the bottom-up path are fused to detect texts of different sizes. Among them, the large feature layer detects small texts, and the small feature layer detects large texts. Then, the feature fusion layer extracts the last layer of features and sends them to the pooling layer, amplifies the features by 1 times, connects the amplified features with the feature map of the previous layer, and performs convolution operations with kernel sizes of 1×1 and 3×3 on the connected feature layer. The results are output to the output layer through 32-kernel, 3×3 convolutions; Finally, the output layer outputs text information and confidence information of the detection box, and the license plate text dataset is passed into the model for training to obtain a license plate text recognition model based on deep learning.

5. A deep learning-based muck truck monitoring system according to claim 4, characterized in that: The data preprocessing module includes constructing a muck truck dataset, collecting data through monitoring of the construction site entrance and exit, extracting frames and removing duplicates from the monitored video stream data, obtaining a muck truck dataset image, and annotating the muck truck license plate information to obtain an OCR model training dataset; The labelimg annotation tool is used to annotate the dump trucks in the dataset. The labels are dump truck (dregs), dump truck empty (empty), dump truck full (full l), dirty dump truck (dirty), clean dump truck (clean), and dump truck license plate (plate). Each image with a labeled target has a corresponding xml label file corresponding to it. Among them, the dump truck label is the data set of the dump truck recognition model, the dump truck compartment empty and dump truck compartment fully loaded labels are the data set of the dump truck loading recognition model, the dirty dump truck and clean dump truck labels are the data set of the dump truck appearance recognition model, and the dump truck license plate label is the label of the dump truck license plate recognition model.

6. The deep learning-based muck truck monitoring system according to claim 1 is characterized in that: The IoT system module includes monitoring equipment, edge smart box, display, mouse and keyboard, and communication equipment; The monitoring equipment is connected to the edge intelligent box for real-time monitoring. After the monitoring can be displayed normally, the deep learning model is deployed. After the deployment is completed, the recognition status is checked through the edge intelligent gateway system. Muck truck recognition, license plate recognition and license plate text recognition, muck truck loading recognition, and muck truck vehicle appearance recognition can all be pushed and viewed in real time. If the license plate is not entered into the system, manual intervention is performed at the entrance of the construction site.

Images