Sanitation work evaluation method based on MRS-YOLO model

Abstract

The invention provides a sanitation work evaluation method based on an MRS-YOLO model, and belongs to the field of machine vision. Firstly, an existing target detection training method SlideLoss is improved, a SlideLossIOU method is provided, and the small target recognition capability is remarkably improved. Secondly, a multi-dimensional convolution mechanism and a dynamic convolution mechanism are adopted, a YOLO model feature extraction method is optimized, and model robustness is enhanced; and finally, a Transform mechanism is introduced, so that the model detection precision and the feature utilization rate are further improved, and meanwhile, the calculation cost is reduced. Compared with a traditional detection method, the garbage can be detected, and a scientific and objective environmental sanitation work evaluation method can be provided by calculating and analyzing a garbage identification result. The MRS-YOLO model has better robustness, higher reasoning speed and higher detection precision, high-efficiency and low-cost intelligent garbage classification is realized, and reliable data support is provided for environmental sanitation work management.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision, and in particular relates to a sanitation work evaluation method based on an MRS-YOLO model. Background Art

[0002] Against the backdrop of rapid global economic development and continuous technological progress, environmental issues have become more urgent, and sanitation work evaluation has become a trend. Garbage sorting, detection and recycling, and cleanliness evaluation are the prerequisites for sanitation work evaluation. However, garbage sorting is still a huge challenge, with problems such as heavy tasks, a wide variety of types, and waste of resources. At present, the mainstream garbage sorting model methods have problems such as low efficiency, high cost, and low recycling rate, which seriously hinder the process of garbage recycling. In this context, there is an urgent need for an intelligent and automated technology that can improve the sorting speed while maintaining high accuracy, so as to better realize garbage recycling and protect the environment.

[0003] At present, garbage detection and recycling methods mainly rely on target detection algorithms to identify garbage. The current target detection algorithms are mainly divided into two types: Two-stage and One-stage. Among them, Two-stage algorithms such as R-CNN, Mask R-CNN, Faster-RCNN, etc., although they have achieved certain results in the field of target detection, have many shortcomings in garbage detection and classification applications. Compared with traditional Two-stage algorithms, One-stage algorithms represented by YOLO, SSD, CornerNet, etc. have the ability to detect multiple targets in an image at one time, which is crucial for classification systems that need to quickly process large amounts of garbage. However, the current One-stage algorithm still needs to be improved in terms of small target detection and model generalization capabilities.

[0004] In addition, there are various problems with traditional methods of garbage sorting and cleanliness evaluation. For example, although near-infrared spectroscopy (NIRS) technology is effective in some cases, it has difficulty distinguishing samples with similar chemical compositions (such as PET and PET-G). In recent years, models such as EfficientNet and YOLO have been used to improve the accuracy and efficiency of garbage sorting using deep learning and computer vision technologies. Among the existing garbage sorting technologies, image recognition models based on deep learning show high potential. For example, by tuning the EfficientNet-B0 model, efficient region-specific garbage sorting can be achieved, greatly improving classification accuracy and computational efficiency. At the same time, the smart trash can system that combines artificial intelligence and the Internet of Things (AIoT) has also significantly improved the efficiency and accuracy of garbage management.

[0005] In summary, there are many shortcomings in the existing technologies in garbage identification and cleanliness evaluation, and there is an urgent need to develop more advanced and efficient technologies to meet the needs of urban governance and environmental protection. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a sanitation work evaluation method based on the MRS-YOLO model; first, the present invention proposes a new garbage detection model MRS-YOLO, based on the existing target detection training method SlideLoss, the present invention improves it, and proposes a method SlideLoss_IOU method that can improve the ability to recognize small targets; secondly, in view of the problem of poor robustness of the YOLO model, a multi-dimensional convolution mechanism and a dynamic convolution mechanism are used to improve the original feature extraction method, and finally a Transformer mechanism is introduced to further enhance the model's ability to detect various garbage, and improve the utilization rate of features while reducing the computational cost to improve the detection accuracy. In addition, compared with the traditional detection method that can only detect garbage simply, the present invention also evaluates the sanitation work by calculating and analyzing the garbage recognition results. The garbage detection model MRS-YOLO proposed by the present invention has better robustness, faster reasoning speed and higher detection accuracy, and can achieve high-efficiency and low-cost intelligent garbage classification. At the same time, the evaluation method can also provide more reliable data support for the management of sanitation work. The invention aims to improve the accuracy and efficiency of garbage detection and provide a scientific and objective sanitation work evaluation method.

[0007] The technical solution of the present invention is as follows:

[0008] A sanitation work evaluation method based on the MRS-YOLO model is proposed. The MRS-YOLO model is used to detect small targets and identify multi-scale garbage. The specific steps are as follows:

[0009] Step 1: Use a camera to capture and pre-process the video stream data in the target area;

[0010] Step 2: Build and train the garbage detection model MRS-YOLO to obtain the final garbage detection model;

[0011] Step 3: Extract image frames from the camera video stream of the target area, use the trained garbage detection model MRS-YOLO to detect garbage in each image of the video stream, identify the amount and type of garbage in each image, and classify the identified garbage;

[0012] Step 4: Calculate the cleanliness index based on the amount of garbage and the coverage area, and evaluate the performance of sanitation work through the set evaluation criteria.

[0013] Furthermore, the specific process of step 1 is as follows: install a camera in the target area to ensure that the camera's field of view covers every corner of the entire area to avoid blind spots; the camera regularly acquires video stream data, and stores the video stream data in real time or in batches to a local server or a cloud server; then, use an automated script to extract image frames from the video stream, and perform segmentation preprocessing on the image to determine the specific location and area of ​​each type of garbage in each image; the segmentation preprocessing process includes image denoising, enhancement and segmentation; then, construct the preprocessed image data into an original data set, and divide it into a training set and a test set according to a ratio of 3:7; the training set is used for model training, and the test set is used for model verification.

[0014] Furthermore, the specific process of step 2 is as follows:

[0015] Step 2.1, select the model framework;

[0016] The YOLOv8 model is selected as the basic framework. The YOLOv8 model consists of two parts: the backbone network and the head and neck network. The photo image of size B×H×W×C is input into the backbone network, where B is the batch size; H is the image height; W is the image width; and C is the number of channels. The backbone network part is used for feature extraction, including convolutional layers and bottleneck blocks, as well as spatial pyramid pooling layers; the head and neck network part is used to process feature maps and predict results, including upsampling layers, connection layers, and convolutional layers, as well as convolutional layers for predicting regression box offsets and category probabilities.

[0017] Step 2.2, establish a small target detection strategy;

[0018] The penalty weight threshold selection method of the original target detection training method SlideLoss is improved. Compared with the original fixed threshold, a SlideLoss_IOU method is proposed. The SlideLoss_IOU method optimizes the choice of weights for the loss function by dynamically adjusting the penalty weight threshold, thereby improving the detection ability of small targets. The principle is as follows:

[0019]

[0020]

[0021] μ = IOUmean;

[0022] Among them, IOUmean represents the calculated intersection-super-union mean; Refers to the unupdated value of the previous generation; d is the decay factor; represents the automatically calculated intersection super-union value, which is used to dynamically adjust the loss function; decay is a parameter that is manually set to control the decay rate; x is the number of training iterations; τ is the time constant; μ is the penalty weight threshold; according to the distribution of different types of samples in the training set, the penalty weight threshold μ is updated by the calculated IOUmean;

[0023] Step 2.3, improve feature extraction method;

[0024] The original feature extraction method of YOLO is improved, and a new feature extraction module CP is proposed; on the premise of retaining the original C2f core function of YOLO, a multi-dimensional pooling convolution mechanism is used to replace the original single convolution module, and then a dynamic convolution mechanism is proposed to adopt different convolution strategies for different feature maps obtained by convolution; specifically, in the convolution part, the new feature extraction module CP adds a multi-dimensional pooling convolution mechanism for feature extraction, and the input feature Figure X Perform selective pooling operation; the process of selective pooling operation is: first, perform dimensional transformation to obtain feature maps of three different perspectives, then perform different pooling operations on the feature maps of different perspectives to obtain different pooling results, then linearly combine the pooling results, and add bias to prevent overfitting. The formula is as follows:

[0025] PoolingOut=AvgPooling(X)+maxPooling(X)+stdPooling(X);

[0026]

[0027] Out=AvgPooling(X)×ω1+maxPooling(X)×ω2+stdPooling(X)×ω3+N;

[0028]

[0029] Among them, PoolingOut is the output of the multi-dimensional pooling convolution mechanism; AvgPooling(·) is average pooling, which is used to calculate the average value of the feature map area; maxPooling(·) is maximum pooling, which is used to select the maximum value of the feature map area; stdPooling(·) is standard deviation pooling, which is used to calculate the standard deviation of the feature map area; Out is the feature map output by the multi-dimensional pooling convolution mechanism; N is the bias; is the number of pooling operations; Sigmoid(·) is the Sigmoid activation function; x is the value input to the sigmoid activation function; ω1, ω2, ω3 are the weights of different pooling operations;

[0030] Next, a dynamic convolution mechanism is added to the feature extraction. The implementation of dynamic convolution is based on the dynamic feature map cutting strategy of the input feature standard deviation. The feature distribution is obtained by obtaining the standard deviation of each feature map, and the number of cuts is dynamically determined according to the standard deviation threshold obtained by training. The standard deviation calculation principle is as follows:

[0031]

[0032]

[0033]

[0034] Among them, μ i is the mean value of the feature map corresponding to channel i; J is the number of feature map matrices corresponding to channel i; x ij is the j-th feature map matrix corresponding to channel i; is the variance calculated for channel i; σ i is the standard deviation of channel i;

[0035] Step 2.4: Optimize the RepViT network architecture;

[0036] An improved Transformer visual model RepViT combined with re-parameterization technology is proposed; in the architecture of the RepViT module, the feature maps obtained by convolution of convolution kernels of different sizes are spliced ​​to obtain feature maps with rich features. The input image is first extracted by the 3×3 convolution kernel of the initial feature extraction module of the starting layer, and its spatial dimension is halved; in each of the following first to fourth feature extraction stages, one or more RepViT modules are included for calculation; between two adjacent stages, the spatial dimension of the image is further reduced and the depth is increased through the downsampling step; the downsampling process is: the feature map with a scale of B×C1×H / 4×W / 4 is sent to the repeated convolution block, the 3×3 convolution block with a stride of 2, the 3×3 convolution block, and the feedforward neural network in turn, and the feature map with a scale of B×C2×H / 8×W / 8 is output. C1 and C2 are the artificially set channel numbers, that is, the depth of the feature map; in the feedforward neural network, the feature map is sequentially subjected to 1×1 convolution, activation function activation, 1×1 convolution, and finally feature fusion; in the fourth feature extraction stage, the feature map is sent to the module RepViTSEBlock and the re-parameterized visual transformer module RepViBlock for further feature extraction; in the module RepViTSEBlock, the feature map is respectively subjected to 3×3 convolution blocks and 1×1 convolution blocks, and then the two results are fused and sent to the module SE, and finally to the feedforward neural network; in the RepViT module, the feature map is respectively subjected to 3×3 convolution blocks and 1×1 convolution blocks, and then the two results are fused and sent to the feedforward neural network; after completing the fourth feature extraction stage, the result is sent to the pooling layer, and the pooling result is sent to the classification module for classification;

[0037] Step 2.5: Perform model training; the garbage detection model MRS-YOLO learns to identify garbage on the training set, and tests it on the test set. Based on the test results, the training model with the best performance is selected as the trained model.

[0038] Furthermore, the specific process of step 4 is as follows:

[0039] Step 4.1: Detect the garbage in the area to be identified by using the proposed garbage detection model MRS-YOLO, and count the amount and type of garbage in the area;

[0040] Step 4.2: Set different scores for different types of garbage in the area and count the number; obtain a total score table containing all types of garbage and scores based on big data statistics, and obtain the score corresponding to each type of garbage from the total score table;

[0041] Step 4.3: Multiply the score and quantity to get the score of each type of garbage, and add up the scores of all types of garbage to get the total score S. total ;

[0042] Step 4.4: Calculate the actual total area A of the target area covered by the acquisition device through the optical parameters of the camera and its angular position parameters with the target area. The calculation process is:

[0043] Step 4.4.1. Calculate the scale factor R of the calibration object:

[0044]

[0045] Where S is the actual size of the calibration object; P is the pixel size of the calibration object in the photo;

[0046] Step 4.4.2, calculate the actual size corresponding to the pixel size in the photo; for the plane perpendicular to the camera, the scale factor between the actual size and the pixel size is directly obtained according to the calculation formula of the scale factor of the calibration object. If there is a shooting angle, the influence of the angle needs to be considered;

[0047] Step 4.4.3, considering the shooting distance and focal length, use the principle of similar triangles to calculate the effect of the distance from the camera to the target area on the size. The calculation formula is as follows:

[0048]

[0049]

[0050] Among them, W actual is the actual width of the target area in the photo; H actual is the actual height of the target area in the photo; W is the width of the target area in the photo; H is the actual height of the target area in the photo; d is the distance from the camera to the target area; f is the focal length of the camera;

[0051] Step 4.4.4: When the multiple shooting angles are not vertical, consider the effect of the shooting angle on the actual size; assuming the shooting angle is θ, the projection length of the target area will become shorter; the calculation formula is as follows:

[0052] W adjusted =W actual ×cos(θ);

[0053] H adjusted =H actual ×cos(θ);

[0054] Among them, W actual H is the adjusted width of the target area in the photo; actual Adjust the height of the target area in the photo;

[0055] Step 4.4.5: Calculate the actual area of ​​the target area using the following formula:

[0056]

[0057] Step 4.5: Calculate the cleaning index CI based on the total score and the actual area of ​​the target area. The formula is:

[0058]

[0059] Among them, α is a tuning parameter used to control the sensitivity of the cleaning index;

[0060] Step 4.6, set the evaluation standard for sanitation work; when the cleanliness index>0.8, the sanitation work is judged to be qualified, indicating that the cleanliness has reached an excellent level; when 0.5<cleanliness index≤0.8, the sanitation work is judged to need improvement, indicating that the cleanliness is medium, but there is still room for improvement; when the cleanliness index≤0.5, the sanitation work is judged to be unqualified, indicating that the cleanliness is poor and improvement measures need to be taken immediately;

[0061] Step 4.7. Regularly calculate and record the cleanliness index to evaluate the performance of sanitation work; regularly calculate and record the cleanliness index to form a systematic sanitation work evaluation report; the specific process is as follows: extract image frames from the camera video stream regularly every day or week, calculate the cleanliness index, record the cleanliness index calculated each time, and summarize it to generate time series data; by comparing the changes in the cleanliness index in different time periods, evaluate the effectiveness and continuous improvement of sanitation work, generate a cleanliness report, and put forward corresponding improvement suggestions.

[0062] Beneficial technical effects brought by the present invention:

[0063] The design and optimization of the MRS-YOLO model of the present invention significantly improves the accuracy and efficiency of garbage detection, which is of great significance for actual garbage classification and environmental protection. First, the SlideLoss_IOU training method improves the detection capability of uncommon garbage types, ensuring that various types of garbage can be detected more comprehensively in practical applications. Secondly, the multi-dimensional convolution method and the dynamic convolution method significantly enhance the feature extraction capability of the model, enabling the model to more accurately identify and classify garbage, thereby improving the overall detection performance. In order to better handle complex garbage detection tasks and make full use of global features, the Transformer mechanism is introduced in the model detection head part, so that the model can better handle long-distance dependencies, and while reducing the computational cost and time, improve the detection effect of the model in complex scenes. These improvements not only improve the detection accuracy and speed of the model, but also enhance its applicability in actual scenarios, providing more reliable technical support for intelligent garbage classification and environmental protection. Through the above model improvements and optimizations, the present invention aims to promote the development of the YOLO model in garbage detection, improve the feature extraction efficiency and robustness of the model, and provide a more effective solution for various garbage detection tasks. Improve the sorting speed while maintaining high precision to better achieve waste recycling and protect the environment.

[0064] The present invention also proposes a sanitation work evaluation method, which calculates the cleanliness index based on the amount of garbage and the coverage area, and realizes the sanitation work evaluation through the set evaluation criteria. The present invention can evaluate the performance of sanitation work more scientifically and objectively, and can perform differentiated treatment according to the impact of different types of garbage on cleanliness. This not only improves the accuracy of garbage detection, but also provides more reliable data support for the management of sanitation work. Through the design and optimization of the MRS-YOLO model, the present invention has important application value in improving the efficiency of garbage classification and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of the sanitation work evaluation method based on the MRS-YOLO model of the present invention.

[0066] Figure 2 It is a network structure diagram of the prior art YOLOv8 model.

[0067] Figure 3 This is a weight change diagram of the SlideLoss method of the prior art YOLOv8 model.

[0068] Figure 4 This is a weight change diagram of the SlideLoss_IOU training method in the MRS-YOLO model proposed in the present invention.

[0069] Figure 5It is a structural diagram of the C2f feature extraction module in the prior art.

[0070] Figure 6 This is a structural diagram of the feature extraction module in the MRS-YOLO model proposed in the present invention.

[0071] Figure 7 This is a diagram of the feature extraction step segmentation method of the C2f module in the prior art.

[0072] Figure 8 This is a diagram of the feature extraction step segmentation method of the improved C2f module integrated with SDC proposed in the present invention.

[0073] Fig. 9 This is a structural diagram of the RepVit module in the prior art.

[0074] Fig.10 This is the network structure diagram of the MRS-YOLO model proposed in the present invention.

[0075] Fig.11 This is a diagram of the experimental results of the MRS-YOLO model in urban corner garbage recognition in the experiment of the present invention.

[0076] Fig.12 This is a graph showing the experimental results of the prior art YOLOv8 model in urban corner garbage recognition in the experiment of the present invention.

[0077] Fig.13 This is a diagram of the experimental results of the MRS-YOLO model in identifying garbage near the wall in the experiment of the present invention.

[0078] Fig.14 This is a graph showing the experimental results of the prior art YOLOv8 model in identifying garbage near a wall in the experiment of the present invention.

[0079] Fig.15 This is a comparison chart of the accuracy standard deviation of the MRS-YOLO model and other models in the experiment of the present invention.

[0080] Fig.16 This is a comparison chart of the accuracy of the MRS-YOLO model and other models in different garbage classifications in the experiment of the present invention.

[0081] Fig.17 This is an example diagram of the cleanliness index evaluation of the MRS-YOLO model in an actual urban corner environment in the experiment of the present invention.

[0082] Fig.18 This is an example diagram of the cleanliness index evaluation of the MRS-YOLO model in an actual wall-side garbage environment in the experiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0084] like Figure 1 As shown, the present invention proposes a sanitation work evaluation method based on the MRS-YOLO model, which performs small target detection and multi-scale garbage recognition through the MRS-YOLO model, and specifically includes the following steps:

[0085] Step 1: Use the camera to capture the video stream data in the target area and perform preprocessing. The specific process is as follows:

[0086] Cameras should be installed reasonably in the target area to ensure that the camera's field of view covers every corner of the entire area to avoid blind spots. The camera should regularly acquire video stream data and store the video stream data in real time or batches to a local server or cloud server to ensure data security and accessibility. Subsequently, an automated script is used to extract image frames from the video stream and perform segmentation preprocessing on the images to determine the specific location and area of ​​each type of garbage in each image. The segmentation preprocessing process includes image denoising, enhancement, and segmentation to improve recognition accuracy. Next, the preprocessed image data is constructed into an original dataset and divided into a training set and a test set according to a ratio of 3:7 to ensure the diversity and representativeness of the dataset. The training set is used for model training, and the test set is used for model validation to evaluate its performance and generalization ability.

[0087] Step 2: Build and train the garbage detection model MRS-YOLO;

[0088] The present invention mainly improves small target detection, feature extraction, and model robustness, and proposes a garbage detection model MRS-YOLO, an improved SlideLoss_IOU training method, etc. The present invention selects the YOLOv8 model as the basic framework, constructs and trains the garbage detection model MRS-YOLO; uses the preprocessed training set for model training, uses the improved SlideLoss_IOU training method to improve the detection capability of small target garbage, and uses multi-dimensional convolution and dynamic convolution methods to improve the robustness of the model.

[0089] The following will introduce the specific work from four parts: model framework, small target detection, improved feature extraction method and Transformer mechanism.

[0090] Step 2.1, select the model framework;

[0091] In order to make the model more practical and lightweight, the present invention chooses to use the latest YOLOv8 model in the prior art as the basic framework. The main difference between YOLOv8 and YOLOv5 is that the C3 module is replaced by the C2f module based on the CSP idea; YOLOv8 adopts the dynamic label matching algorithm TaskAlignedAssigner for matching strategy.

[0092] The C2f module draws on the ELAN idea in YOLOv7, combining C3 and ELAN into the C2f module, so that YOLOv8 can obtain richer gradient flow information while ensuring lightweight. At the end of the backbone network, the most popular spatial pyramid pooling layer SPPF module is still used, and three maximum pooling layers Maxpools with a size of 5×5 are continuously passed. Then, each layer is connected in series, thereby ensuring the accuracy of objects at various scales while ensuring the lightweight of the model.

[0093] In the head and neck network (Head + Neck), the feature fusion method adopted by YOLOv8 is still PAN-FPN, which strengthens the fusion and utilization of information from feature layers of different scales. YOLOv8 uses two upsampling and multiple C2f modules and the final decoupled head structure to form the neck module. YOLOv8 uses the idea of ​​head decoupling in YOLOx for the last part of the neck. It combines confidence and regression boxes to achieve a new level of accuracy.

[0094] YOLOv8 can support all versions of YOLO and can switch between different versions at will. It can also run on various hardware platforms (CPU-GPU) and has strong flexibility.

[0095] Figure 2The network structure of the YOLOv8 model is shown. The structure consists of two parts: the backbone network Backbone and the head and neck network. It is specifically used for the garbage recognition method based on the MRS-YOLO model. The photo image of size B×H×W×C is input into the backbone network, where B is the batch size; H is the image height; W is the image width; and C is the number of channels. In the Backbone part, the model includes multiple convolutional layers and bottleneck blocks for feature extraction, including a convolutional layer CBS with a stride S of 2 and a convolution kernel K of 3, a bottleneck block C2f containing 3 or 6 submodules n, and a spatial pyramid pooling layer SPPF. In the head and neck network part, it is mainly used to process feature maps and make result predictions, including an upsampling layer, a connection layer R, a convolutional layer CBS with a stride S of 1 and a convolution kernel K of 3, and a 1×1 convolutional layer Conv_4 for predicting regression box offsets and category probabilities. Through the combination of these layers and feature processing, the YOLOv8 model can improve the detection accuracy of small and multi-scale targets while maintaining high efficiency and lightweight. This structure has been applied and improved in the MRS-YOLO model to better meet the needs of garbage identification and improve the model's robustness, detection accuracy, and reasoning speed. Figure 2 In the formula, reg_max is the adjustment factor; num_class is the number of classes;

[0096] Step 2.2, establish a small target detection strategy;

[0097] In the real-world garbage detection task, it is difficult to detect small-target garbage, and the detection accuracy is low. However, small-target garbage accounts for a high proportion of domestic garbage. Therefore, improving the accuracy of small-target detection becomes particularly important in garbage detection tasks. The original target detection training method SlideLoss is used to solve the problem of extremely uneven distribution of data sample types. That is, some types of garbage (such as plastic bottles or paper) appear frequently in the data set, while some rare types of garbage (such as hazardous waste) are relatively rare. By increasing the penalty weight of difficult-to-detect samples, the model's attention to these samples is increased. The basic principle of the existing target detection training method SlideLoss is shown in formulas (1) and (2):

[0098]

[0099]

[0100] Among them, SlideLoss(·) is the existing target detection training method SlideLoss; y is the true value; is the predicted value of the model; u is the average value of the intersection and superunion; ω(·) is the weight function; L(·) is the loss function; μ is the penalty weight threshold; is the calculated intersection super-union; the calculated intersection super-union is compared with μ to select different loss function weights; the SlideLoss method adjusts the sample loss according to the IOU value of each sample, effectively enhancing the model's ability to recognize difficult-to-detect and classify samples. To put it more intuitively, Figure 3 As shown, Figure 3 The weight change curve of the SlideLoss method is shown, which optimizes the loss function of the model by dynamically adjusting the loss weight based on the intersection-super-union (IOU). Figure 3 In the figure, the horizontal axis represents the sample score, the vertical axis represents the scoring function value, and the illustrated area is divided into the negative sample area and the positive sample area. In the negative sample area, when the sample score is in the range of 0 to μ-0.1, the scoring function value remains constant 1, indicating that the processing of these samples remains unchanged. In the positive sample area, when the sample score is in the range of μ to positive infinity, the scoring function value changes from e 1-μ It gradually decays to 1, indicating that the penalty weight for positive samples decreases as the score increases. This method effectively enhances the model's ability to identify samples that are difficult to detect and classify, especially improving the detection accuracy and flexibility of rare types of garbage, further improving the performance of the overall model.

[0101] On this basis, the present invention improves the penalty weight threshold selection method of the original target detection training method SlideLoss. Compared with the original fixed threshold, a SlideLoss_IOU method is proposed. The SlideLoss_IOU method optimizes the selection of weights for the loss function by dynamically adjusting the penalty weight threshold, thereby improving the detection capability of small targets. The principle is shown in formulas (3) to (5):

[0102]

[0103]

[0104] μ = IOUmean (5);

[0105] Among them, IOUmean represents the calculated intersection-super-union mean; refers to the unupdated value of the previous generation, and d is the decay factor; Represents the automatically calculated intersection super-union value, which is used to dynamically adjust the loss function. This variable is only used for calculation and does not participate in echelon descent or gradient descent. decay is a parameter that is manually set to control the decay rate. x is the number of training iterations. τ is a time constant that controls the decay rate. Then the calculated intersection super-union mean IOUmean is assigned to As the new threshold. μ is the penalty weight threshold; according to the distribution of different types of samples in the training set, the penalty weight threshold μ is updated by the calculated IOUmean. Figure 4 The weight change curve of the SlideLoss_IOU method in the MRS-YOLO model is shown. Figure 4 In the figure, the horizontal axis represents the loss function threshold, the vertical axis represents the loss function weight, and the illustrated area is divided into a complex sample area and a simple sample area. In the complex sample area, when the loss function threshold is in the range of 0 to μ-0.1, the loss function weight remains constant 1, indicating that the processing of these complex samples remains unchanged. In the simple sample area, when the loss function threshold is in the range of μ to positive infinity, the loss function weight changes from e 1-μ It gradually decays to 1, indicating that the penalty weight for simple samples decreases as the score increases. The SlideLoss_IOU method optimizes the choice of weights in the loss function by penalizing the weight threshold to improve the detection ability of small targets. This method introduces a dynamic threshold to pay more attention to the detection of small targets, improving the flexibility and detection ability of the model in detecting rare garbage types and complex occlusion scenarios. Figure 4 The arrows in the middle show how the loss function weights change between complex samples and simple samples.

[0106] The SlideLoss method solves the problem of unbalanced distribution of training sample categories by using a fixed penalty weight threshold. The SlideLoss_IOU method proposed in the present invention can pay more attention to the detection of small targets by introducing a dynamic penalty weight threshold, and will also have better flexibility in the face of occlusion scenes, thereby improving the detection capability of the model.

[0107] Step 2.3, improve feature extraction method;

[0108] In order to solve the problem of poor robustness of YOLO in garbage detection, the present invention improves the original feature extraction method of YOLO and proposes a new feature extraction module (CP). On the premise of retaining the original C2f core function of YOLO, a multi-dimensional pooling convolution method is used to replace the original single convolution module, and then a dynamic convolution mechanism is proposed to adopt different convolution strategies for different feature maps obtained by convolution. The above improvements ensure that the model of the present invention can maintain good target detection capabilities in various types of garbage detection, and is more flexible in processing different feature maps.

[0109] Specifically, in the convolution part, compared with C2f using the Conv module for feature extraction, the present invention introduces a new feature extraction module. The new feature extraction module adds a multi-dimensional pooling convolution mechanism (i.e., a multi-scale channel attention mechanism module, PoolAttention) for feature extraction. The original convolution structure of C2f and the new feature extraction module are compared. Figure 5 and Figure 6 shown.

[0110] Figure 5 The structure of the existing C2f feature extraction module is shown. The C2f module draws on the ELAN idea in YOLOv7 and combines C3 and ELAN into the C2f module, so that YOLOv8 can obtain richer gradient flow information while ensuring lightweight. Specifically, the C2f module processes the input feature map multiple times through the combination of the convolutional layer Conv2d and the bottleneck block BatchNom2d, extracts multi-scale feature information, and significantly enhances the model's ability to capture subtle features through channel expansion and feature re-decomposition. The SiLU activation function is used in the C2f module.

[0111] Figure 6 The structure of the new feature extraction module in the MRS-YOLO model is shown. Figure 6 The sum operation is feature fusion. This module is improved based on the C2f of YOLOv8 and introduces a multi-dimensional pooling convolution mechanism for feature extraction. Compared with the traditional C2f module, the idea of ​​the module of the present invention is to convert the input features into Figure X Perform selective pooling operation; the process of selective pooling operation is: first, perform dimensional transformation to obtain feature maps of three different perspectives, then perform different pooling operations on the feature maps of different perspectives to obtain different pooling results, then linearly combine the pooling results, and add bias to prevent overfitting. The formula is as follows:

[0112] PoolingOut=AvgPooling(X)+maxPooling(X)+stdPooling(X)(6);

[0113]

[0114] Out=AvgPooling(X)×ω1+maxPooling(X)×ω2+stdPooling(X)×ω3+N(8);

[0115]

[0116] Among them, PoolingOut is the output of the multi-dimensional pooling convolution mechanism; AvgPooling(·) is average pooling, which is used to calculate the average value of the feature map area; maxPooling(·) is maximum pooling, which is used to select the maximum value of the feature map area; stdPooling(·) is standard deviation pooling, which is used to calculate the standard deviation of the feature map area. N is the bias; Out is the feature map output by the multi-dimensional pooling convolution mechanism; is the number of pooling operations, usually 3; Sigmoid(·) is the Sigmoid activation function; x is the value input to the Sigmoid activation function; ω1, ω2, ω3 are the weights of different pooling operations, and their value ranges are all in [0,1].

[0117] This design greatly reduces the computational cost of subsequent operations. By fusing different types of pooling operations and introducing an adaptive attention mechanism, the PoolAttention module can make more detailed adjustments and optimizations to different dimensions of the feature map, thereby extracting richer and more effective feature representations.

[0118] In the dynamic convolution part, the C2f method divides the feature map into multiple parts before constructing the residual block. Each part can be regarded as a feature subset between different channels. This helps the network to better learn the feature relationship between different channels and improves the flexibility and expression ability of the network. The present invention further develops on this idea by adding a dynamic convolution mechanism to feature extraction. The feature distribution of each feature map obtained during the convolution process is obtained by the standard deviation. The number of divisions is dynamically determined according to the standard deviation threshold obtained through training. The standard deviation calculation principle is shown in formulas (10) to (12):

[0119]

[0120]

[0121]

[0122] in, is the mean value of the feature map corresponding to channel i; J is the number of feature map matrices corresponding to channel i; x ij is the j-th feature map matrix corresponding to channel i; is the variance calculated for channel j; σ i is the standard deviation of channel i; in general, the first step is to calculate the average value of the feature map corresponding to channel i, then calculate its variance for channel i, and finally take the square root to obtain the standard deviation σ corresponding to channel i i .

[0123] The improved C2f structure of the present invention is compared with the original C2f structure. Figure 7 and Figure 8 shown.

[0124] YOLOv8's C2f (Channel to Feature) technology is inspired by the ELAN mechanism. It combines the ELAN mechanism with the C3 module, and significantly enhances the model's ability to capture subtle features through channel expansion and feature decomposition. Figure 7 It is the C2f structure of YOLOv8. The convolution layer Convs performs convolution to obtain a new feature map in the form of (B, 2×C, H, W), and then enters the feature block Chunk for segmentation to obtain a fixed number of 2 feature maps in the form of (B, C, H, W), and then sends them to the bottleneck layer Bottleneck;

[0125] The present invention mainly designs spatial dynamic convolution SDC, Figure 8 The present invention proposes an improved C2f structure diagram combined with spatial dynamic convolution SDC. The implementation of spatial dynamic convolution SDC is based on a dynamic feature map cutting strategy of the input feature standard deviation. The standard deviation is calculated for each input feature map, and the number of segments is dynamically adjusted according to the threshold. This enables the model to dynamically adjust its processing flow according to changes in input data, making it more flexible to process different situations. Figure 8 In the convolution layer Convs, the convolution operation is performed to obtain a feature map containing deeper features, and then enters the feature block Chunk to dynamically split according to the most appropriate number of splits calculated in the above steps to obtain the corresponding number The feature map is then sent to the bottleneck layer; at the same time, the spatial dynamic convolution SDC adopts a dynamic feature map cutting strategy based on the standard deviation of the input features.

[0126] By setting a suitable threshold, the number of features contained in the feature map is divided into three categories: fewer features, general features, and rich features. The slicing strategy will be adjusted accordingly to match the corresponding number of bottleneck iterations. This method significantly enhances the robustness of the model, enabling it to flexibly respond to different situations, and dynamically adjust the network structure according to the characteristics of the input data to better adapt to different scenarios and task requirements. Specifically, the standard deviation of the feature map is used to determine whether the features it contains are rich. If the features are rich, the feature map is divided into more simple feature map parts, and Bottleneck is performed multiple times to better extract the features; if the features are fewer, the number of segmentations is reduced to retain more global features and save computational costs to a certain extent.

[0127] YOLOv8's C2f (Channel to Feature) technology is inspired by the ELAN mechanism, which combines the ELAN mechanism with the C3 module. Through channel expansion and feature re-decomposition, the model's ability to capture subtle features is significantly enhanced. The design of the present invention implements a dynamic feature map cutting strategy SDC based on the standard deviation of the input features, calculates the standard deviation for each input feature map, and dynamically adjusts the number of cuts according to the threshold, so that the model can dynamically adjust its processing flow according to changes in input data, making it more flexible to handle different situations. This strategy not only optimizes the feature extraction process, but also improves the robustness and generalization ability of the model in the face of different types of data. Through these designs, the present invention aims to further push the boundaries of the YOLO model in terms of feature extraction efficiency and robustness in garbage detection, and provide new solutions for garbage detection tasks with a wide range of data categories.

[0128] Step 2.4: Optimize the RepViT network architecture;

[0129] In order to better handle complex garbage detection tasks and make full use of global features, the present invention refers to an improved Transformer visual model RepViT combined with re-parameterization technology. The computational complexity and memory requirements of the traditional Transformer model increase with the square of the sequence length. It is not friendly to tasks with complex scenes and a wide variety of targets. The RepViT module is an improved Transformer visual model combined with re-parameterization technology. It reduces computational cost and time by optimizing its structure, making the Transformer model more suitable for object detection tasks. In the architecture of the RepViT module, its principles are as follows: Fig. 9As shown in the figure, the feature maps obtained by convolution of different sizes of convolution kernels are concatenated to obtain a feature map with richer features. The input image is first extracted by the 3×3 convolution kernel of the initial feature extraction module Stem of the starting layer, and its spatial dimension is halved; the stride of the convolution kernel is 2, and Act is the corresponding activation function. In each subsequent feature extraction stage (stage1 to stage4), one or more RepViT modules are included for calculation. Between two adjacent stages, the spatial dimension of the image is further reduced and the depth is increased through the downsampling step, which helps to focus on higher-level features while reducing computational overhead. The downsampling process is: the feature map with a scale of B×C1×H / 4×W / 4 is sequentially sent to the repeated convolution block RepViBlock, the 3×3 convolution block DW with a stride of 2, the 3×3 convolution block, and the feedforward neural network FFN, and the output scale is B×C2×H / 8×W / 8. Feature map, where B is the batch size, that is, the number of samples input to the model at one time, C1 and C2 represent the number of channels set manually, that is, the depth of the feature map. In the feedforward neural network FFN, the feature map is sequentially subjected to 1×1 convolution, Act, and 1×1 convolution, and finally feature fusion is performed. In the fourth feature extraction stage stage4, the feature map is sent to the module RepViTSEBlock and the reparameterized visual transformer module RepViBlock for further feature extraction. In the module RepViTSEBlock, the feature map passes through the 3×3 convolution block DW and the 1×1 convolution block DW respectively, and then the two convolution results are fused and sent to the Squeeze-and-Excitation module SE, and finally sent to the feedforward neural network FFN. In the RepViT module, the feature map passes through the 3×3 convolution block DW and the 1×1 convolution block DW respectively, and then the two convolution results are fused and sent to the feedforward neural network FFN. After completing the fourth feature extraction stage, the result is sent to the pooling layer, and the pooling result is sent to the classification module PC module for classification. Fig. 9 Inference is the inference stage. In the inference stage, the model uses the trained parameters to process the newly input image and output the recognition result.

[0130] By integrating the RepViT architecture into YOLOv8, the Transformer's self-attention mechanism can be used to capture long-distance dependencies in images. At the same time, the re-parameterization technology is used to reduce the risk of overfitting on small data sets, further enhancing the robustness and accuracy of the model, providing a more effective solution for complex garbage detection tasks.

[0131] The final MRS-YOLO network structure is as follows Fig.10 shown. Fig.10The overall network structure of the MRS-YOLO model is demonstrated. The model is based on the YOLOv8 architecture and is adapted to the garbage recognition task through a number of improvements. The CP module is used to replace the original C2f module in the backbone network. The CP module is introduced into the neck network and the self-attention mechanism is used to capture long-distance dependencies through the improved Transformer visual model RepViT module, which ultimately improves the model's detection accuracy and adaptability to small targets and complex background garbage.

[0132] Step 2.5: Perform model training; use the garbage detection model MRS-YOLO in the training set D z-train Learn to identify garbage on the test set D z-test The training model with the best performance is tested on the network. According to the test results, the training model with the best performance is selected as the trained model. The trained model can also be trimmed and optimized to convert it into an inference model to obtain the final garbage detection model. For example, the training model with the highest test mAP value (mean average precision) can be selected as the training model with the best performance. The specific process of trimming and optimizing the model is as follows: First, the computational complexity and storage requirements of the model are reduced by pruning redundant network layers and nodes; then, the model weights are converted from floating point numbers to low-precision representations through quantization technology to further reduce computational and memory overheads; finally, the distillation learning method is used to improve the accuracy and performance of the trimmed model by letting the lightweight model learn the output of the large teacher model. Through these steps, an efficient and accurate inference model is finally obtained for garbage identification in practical applications.

[0133] Step 3: Extract image frames from the camera video stream of the target area, use the trained garbage detection model MRS-YOLO to detect garbage in each image of the video stream, identify the amount and type of garbage in each image, and classify the identified garbage.

[0134] Step 4: Calculate the cleanliness index based on the amount of garbage and the coverage area, and evaluate the performance of sanitation work according to the set evaluation criteria; the specific process is as follows:

[0135] Step 4.1: Detect the garbage in the area to be identified by using the proposed garbage detection model MRS-YOLO, and count the amount and type of garbage in the area;

[0136] Step 4.2, set different scores for different types of garbage in the area, and count the number; obtain a total score table containing all types of garbage and scores based on big data statistics, and obtain the score corresponding to each type of garbage from the total score table; for example, set 1 point per piece for kitchen waste, 2 points per piece for metal waste, 0.5 points per piece for paper waste, 1.5 points per piece for plastic waste, etc.;

[0137] Step 4.3: Multiply the score and quantity to get the score of each type of garbage, and add up the scores of all types of garbage to get the total score S. total ;

[0138] Step 4.4: Calculate the actual total area A of the target area covered by the acquisition device through the optical parameters (focal length) of the camera and its angular position parameters with the target area. The unit of A is square centimeters. The calculation process is:

[0139] Step 4.4.1. Calculate the scale factor R of the calibration object:

[0140]

[0141] Where S is the actual size of the calibration object, in centimeters; P is the pixel size of the calibration object in the photo, in pixels;

[0142] Step 4.4.2, calculate the actual size corresponding to the pixel size in the photo; for the plane perpendicular to the camera, the scale factor between the actual size and the pixel size is directly obtained according to the calculation formula of the scale factor of the calibration object. If there is a shooting angle, the influence of the angle needs to be considered;

[0143] Step 4.4.3, considering the shooting distance and focal length, use the principle of similar triangles to calculate the effect of the distance from the camera to the target area on the size. The calculation formula is as follows:

[0144]

[0145]

[0146] Among them, W actual is the actual width of the target area in the photo, in pixels; H actual is the actual height of the target area in the photo, in pixels; W is the width of the target area in the photo, in pixels; H is the actual height of the target area in the photo, in pixels; d is the distance from the camera to the target area, in centimeters; f is the focal length of the camera;

[0147] Step 4.4.4: When the multiple shooting angles are not vertical, consider the effect of the shooting angle on the actual size; assuming that the shooting angle is θ and the unit of the shooting angle is degree, the projection length of the target area will become shorter; the calculation formula is as follows:

[0148] W adjusted =W actual ×cos(θ) (16);

[0149] H adjusted =H actual ×cos(θ) (17);

[0150] Among them, W actual H is the adjusted width of the target area in the photo, in pixels; actual is the adjusted height of the target area in the photo, in pixels;

[0151] Step 4.4.5: Calculate the actual area of ​​the target area:

[0152] A=W adjusted ×J adjusted ;(18);

[0153] Substituting the previous formula into the equation, we get:

[0154]

[0155] The simplified formula is:

[0156]

[0157] To summarize, the complete mathematical formula of A is:

[0158]

[0159] Step 4.5: Calculate the cleaning index CI based on the total score and the actual area of ​​the target area. The formula is:

[0160]

[0161] Here, α is a tuning parameter used to control the sensitivity of the cleaning index.

[0162] Step 4.6, set the evaluation criteria for sanitation work;

[0163] In order to scientifically and objectively evaluate the performance of sanitation work, the specific evaluation standards for sanitation work are set as follows: when the cleanliness index > 0.8, the sanitation work is judged to be qualified, indicating that the cleanliness has reached an excellent level; when 0.5 < cleanliness index ≤ 0.8, the sanitation work is judged to need improvement, indicating that the cleanliness is medium but there is still room for improvement; when the cleanliness index ≤ 0.5, the sanitation work is judged to be unqualified, indicating that the cleanliness is poor and immediate improvement measures need to be taken.

[0164] Step 4.7, regularly calculate and record the cleaning index;

[0165] The cleanliness index is calculated and recorded regularly to form a systematic sanitation work evaluation report. The specific process is as follows: extract image frames from the camera video stream regularly every day or week, calculate the cleanliness index, record the cleanliness index calculated each time, and summarize and generate time series data for long-term monitoring and analysis. By comparing the changes in the cleanliness index in different time periods, evaluate the effectiveness and continuous improvement of sanitation work, generate a cleanliness report, and put forward corresponding improvement suggestions, providing reliable data support and scientific basis for sanitation work management. Through these optimized steps, the performance of sanitation work can be evaluated more scientifically and objectively, and differentiated treatment can be carried out according to the impact of different types of garbage on cleanliness. This not only improves the accuracy of garbage detection, but also provides more reliable data support for the management of sanitation work.

[0166] In order to prove the feasibility and superiority of the present invention, the following experiment was conducted.

[0167] Figure 11-Figure 14 The comparison results of the MRS-YOLO model of the present invention and the prior art YOLO-V8 model in various garbage recognition in different scenarios are shown. The numbers in the figure represent the recognition accuracy. Fig.11 and Fig.12 They are respectively the experimental result diagrams of the MRS-YOLO model of the present invention and the YOLOv8 model of the prior art in the urban corner garbage recognition in the experiment of the present invention. For example, for recyclable metals, the MRS-YOLO model of the present invention can achieve an accuracy of 0.47, while the YOLOv8 model of the prior art can only achieve 0.46. Fig.13 and Fig.14 The figures are respectively the experimental results of the MRS-YOLO model of the present invention and the YOLOv8 model of the prior art in the wall garbage recognition. For example, for recyclable metals, the MRS-YOLO model of the present invention can achieve an accuracy of 0.27, while the YOLOv8 model of the prior art cannot recognize it. Figure 11-Figure 14 It can be seen that the recognition accuracy of the MRS-YOLO model of the present invention is higher.

[0168] In order to verify the effectiveness of the enhancements introduced in the MRS-YOLO model proposed in the present invention, the model of the present invention was compared with other widely recognized garbage detections, and the models were experimented under the same control parameters. All models were evaluated using the same equipment, data sets, and training parameters. The experimental comparison results are shown in Table 1. Among them, YOLOv5n is a lightweight version of the YOLO series, designed for real-time object detection on edge devices. RT-DETR is a real-time detector that combines the powerful features of transformers for efficient target detection. YOLOv8n provides higher detection accuracy and speed and is suitable for various complex scenarios. YOLOv3 uses a multi-scale feature pyramid to improve the detection performance of small targets. YOLOv6n further optimizes the network structure and improves detection speed and accuracy. YOLOv5s is the standard version of YOLOv5, which is both efficient and accurate. Fast-RCNN is a region-based convolutional neural network for accurate target detection and classification. MASK-RCNN adds instance segmentation function on the basis of Fast-RCNN, which can detect objects and generate masks. EfficientDet uses an efficient compound scaling strategy to improve detection accuracy while maintaining high efficiency.

[0169] Table 1 Comparison results of the accuracy of the MRS-YOLO model of the present invention and other models

[0170]

[0171]

[0172] The results show that the backbone network of the proposed model outperforms other backbone networks in terms of mAP under two IOU thresholds, achieving 0.5 Achieved 74.5% and mAP 0.5:0.95 Reached 65.5%.

[0173] By calculating the standard deviation of the detection results of YOLOv3, YOLOv5, YOLOv8 and the model proposed in this invention, we can get insight into the stability level of their detection performance in different categories. The lower the standard deviation, the more consistent and stable the performance, while the higher the standard deviation, the greater the variability of the detection results.

[0174] The present invention compares the MRS-YOLO model with other models in terms of accuracy standard deviation. The comparison results are shown in the following table. Fig.15 As shown. Fig.15In the comparison results, the model proposed by the present invention has the lowest standard deviation of 0.074971. Compared with YOLOv3, YOLOv5 and YOLOv8, the model proposed by the present invention has higher detection stability and consistency levels in different categories. This means that the model of the present invention shows more reliable and robust performance in accurately detecting objects of different categories, which is crucial for applications that need to maintain consistent detection performance in different scenarios. In addition, the standard deviations of YOLOv3, YOLOv8 and the model of the present invention are all on a downward trend, indicating that with the advancement of model architecture and technology, stability and reliability are constantly improving.

[0175] The MRS-YOLO model of the present invention mainly includes three main components: a new C2f-based feature extraction module CP is proposed, a new small target detection strategy SlideLoss_IOU is introduced, and the network structure is optimized by integrating RepViT. In order to verify the effectiveness of each innovation point, an ablation experiment is carried out to compare its performance with that of the original model. The experimental results are shown in Table 2.

[0176] Table 2 Comparison of ablation experiment results

[0177]

[0178] As can be seen from Table 2, when each component is added to the model separately, a significant improvement in the performance of garbage detection can be observed. Specifically, at IOU0.5 and 0.5:0.95, the addition of CP significantly improves the mAP, proving that the feature extraction module CP can effectively improve the efficiency of feature extraction; the addition of SlideLoss_IOU significantly improves the mAP value, proving the effectiveness of SlideLoss_IOU in object detection; the addition of RepViT can significantly improve the accuracy and speed of the model. When all three components are combined, the model achieves the highest mAP score, effectively demonstrating the superiority of the MRS-YOLO model of the present invention.

[0179] Fig.16 The improvement in category accuracy achieved by integrating various modules in the ablation experiment is demonstrated. YOLOv8+CP means adding the CP module to the YOLOv8 model, YOLOv8+Repvit means adding the RepViT module to the YOLOv8 model, and YOLOv8+Slideloss_IOU means adding the SlideLoss_IOU to the YOLOv8 model.

[0180] from Fig.16It can be seen that the MRS-YOLO model consistently outperforms other models, with an overall accuracy of 0.745, a significant improvement of 0.036 over the baseline YOLOv8 model. It is particularly noteworthy that the MRS-YOLO model achieved high accuracy in the categories of non-recyclable garbage, kitchen waste, and recyclable glass, with accuracy rates of 0.814, 0.821, and 0.659, respectively. In the category of recyclable textiles, the MRS-YOLO model performed well, with an accuracy of 0.852, a significant improvement over other models. In addition, the model of the present invention also performs well in detecting small targets such as recyclable metals and recyclable ceramics, thus confirming its effectiveness in small target detection. These results together highlight the excellent performance and effectiveness of the MRS-YOLO model in garbage recognition in various scenarios, and prove its applicability in garbage detection tasks.

[0181] Fig.17 and Fig.18 They are respectively example diagrams of the cleanliness index evaluation of the MRS-YOLO model in an actual urban corner environment and an actual wall-side garbage environment in the experiment of the present invention. Fig.17 and Fig.18 In the above table, the cleanliness indexes calculated by formula (22) are 0.387 and 0.112 respectively. The sanitation work is judged as unqualified, indicating that the cleanliness is poor and improvement measures need to be taken immediately.

[0182] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A sanitation work evaluation method based on the MRS-YOLO model, characterized in that: The MRS-YOLO model is used to detect small targets and identify multi-scale garbage. The specific steps are as follows: Step 1: Use a camera to capture and pre-process the video stream data in the target area; Step 2: Build and train the garbage detection model MRS-YOLO to obtain the final garbage detection model; Step 3: Extract image frames from the camera video stream of the target area, use the trained garbage detection model MRS-YOLO to detect garbage in each image of the video stream, identify the amount and type of garbage in each image, and classify the identified garbage; Step 4: Calculate the cleanliness index based on the amount of garbage and the coverage area, and evaluate the performance of sanitation work through the set evaluation criteria.

2. The sanitation work evaluation method based on the MRS-YOLO model according to claim 1 is characterized in that: The specific process of step 1 is as follows: installing a camera in the target area to ensure that the camera's viewing angle covers every corner of the entire area to avoid blind spots; the camera regularly obtains video stream data and stores the video stream data in real time or in batches to a local server or a cloud server; Subsequently, an automated script was used to extract image frames from the video stream and perform segmentation preprocessing on the images to determine the specific location and area of ​​each type of garbage in each image; The segmentation preprocessing process includes image denoising, enhancement and segmentation; then, the preprocessed image data is constructed into an original data set and divided into a training set and a test set according to a ratio of 3:7; the training set is used for model training, and the test set is used for model verification.

3. The sanitation work evaluation method based on the MRS-YOLO model according to claim 1 is characterized in that: The specific process of step 2 is: Step 2.1, select the model framework; The YOLOv8 model is selected as the basic framework. The YOLOv8 model consists of two parts: the backbone network and the head and neck network. The photo image of size B×H×W×C is input into the backbone network, where B is the batch size; H is the image height; W is the image width; and C is the number of channels. The backbone network part is used for feature extraction, including convolutional layers and bottleneck blocks, as well as spatial pyramid pooling layers; the head and neck network part is used to process feature maps and predict results, including upsampling layers, connection layers, and convolutional layers, as well as convolutional layers for predicting regression box offsets and category probabilities. Step 2.2, establish a small target detection strategy; The penalty weight threshold selection method of the original target detection training method SlideLoss is improved. Compared with the original fixed threshold, a SlideLoss IOU method is proposed. The SlideLoss IOU method optimizes the choice of weights for the loss function by dynamically adjusting the penalty weight threshold, thereby improving the detection ability of small targets. The principle is as follows: μ = IOUmean; Among them, IOUmean represents the calculated intersection-super-union mean; Refers to the unupdated value of the previous generation; d is the decay factor; represents the automatically calculated intersection super-union value, which is used to dynamically adjust the loss function; decay is a parameter that is manually set to control the decay rate; x is the number of training iterations; τ is the time constant; μ is the penalty weight threshold; according to the distribution of different types of samples in the training set, the penalty weight threshold μ is updated by the calculated IOUmean; Step 2.3, improve feature extraction method; The original feature extraction method of YOLO is improved, and a new feature extraction module CP is proposed; under the premise of retaining the original C2f core function of YOLO, a multi-dimensional pooling convolution mechanism is used to replace the original single convolution module, and then a dynamic convolution mechanism is proposed to adopt different convolution strategies for different feature maps obtained by convolution; specifically, in the convolution part, the new feature extraction module CP adds a multi-dimensional pooling convolution mechanism for feature extraction, and performs a selective pooling operation on the input feature map X; the process of the selective pooling operation is: first, the dimension transformation is performed to obtain feature maps of three different perspectives, and then different pooling operations are performed on the feature maps of different perspectives to obtain different pooling results, and then the pooling results are linearly combined, and a bias is added to prevent overfitting. The formula is as follows: PoolingOut=AvgPooling(X)+maxPooling(X)+stdPooling(X); Out=AvgPooling(X)×ω1+maxPooling(X)×ω2+stdPooling(X)×ω3+N; Among them, PoolingOut is the output of the multi-dimensional pooling convolution mechanism; AvgPooling(·) is average pooling, which is used to calculate the average value of the feature map area; maxPooling(·) is maximum pooling, which is used to select the maximum value of the feature map area; stdPooling(·) is standard deviation pooling, which is used to calculate the standard deviation of the feature map area; Out is the feature map output by the multi-dimensional pooling convolution mechanism; N is the bias; is the number of pooling operations; Sigmoid(·) is the Sigmoid activation function; x is the value input to the sigmoid activation function; ω1, ω2, ω3 are the weights of different pooling operations; Next, a dynamic convolution mechanism is added to the feature extraction. The implementation of dynamic convolution is based on the dynamic feature map cutting strategy of the input feature standard deviation. The feature distribution is obtained by obtaining the standard deviation of each feature map, and the number of cuts is dynamically determined according to the standard deviation threshold obtained by training. The standard deviation calculation principle is as follows: Among them, μ i is the mean value of the feature map corresponding to channel i; J is the number of feature map matrices corresponding to channel i; x ij is the j-th feature map matrix corresponding to channel i; is the variance calculated for channel i; σ i is the standard deviation of channel i; Step 2.4: Optimize the RepViT network architecture; An improved Transformer visual model RepViT combined with re-parameterization technology is proposed; in the architecture of the RepViT module, the feature maps obtained by convolution of convolution kernels of different sizes are spliced ​​to obtain feature maps with rich features. The input image is first extracted by the 3×3 convolution kernel of the initial feature extraction module of the starting layer, and its spatial dimension is halved; in each of the following first to fourth feature extraction stages, one or more RepViT modules are included for calculation; between two adjacent stages, the spatial dimension of the image is further reduced and the depth is increased through the downsampling step; the downsampling process is: the feature map with a scale of B×C1×H / 4×W / 4 is sent to the repeated convolution block, the 3×3 convolution block with a stride of 2, the 3×3 convolution block, and the feedforward neural network in turn, and the feature map with a scale of B×C2×H / 8×W / 8 is output. C1 and C2 are the artificially set channel numbers, that is, the depth of the feature map; in the feedforward neural network, the feature map is sequentially subjected to 1×1 convolution, activation function activation, 1×1 convolution, and finally feature fusion; in the fourth feature extraction stage, the feature map is sent to the module RepViTSEBlock and the re-parameterized visual transformer module RepViBlock for further feature extraction; in the module RepViTSEBlock, the feature map is respectively subjected to 3×3 convolution blocks and 1×1 convolution blocks, and then the two results are fused and sent to the module SE, and finally to the feedforward neural network; in the RepViT module, the feature map is respectively subjected to 3×3 convolution blocks and 1×1 convolution blocks, and then the two results are fused and sent to the feedforward neural network; after completing the fourth feature extraction stage, the result is sent to the pooling layer, and the pooling result is sent to the classification module for classification; Step 2.5: Perform model training; the garbage detection model MRS-YOLO learns to identify garbage on the training set, and tests it on the test set. Based on the test results, the training model with the best performance is selected as the trained model.

4. The sanitation work evaluation method based on the MRS-YOLO model according to claim 1 is characterized in that: The specific process of step 4 is as follows: Step 4.1: Detect the garbage in the area to be identified by using the proposed garbage detection model MRS-YOLO, and count the amount and type of garbage in the area; Step 4.2: Set different scores for different types of garbage in the area and count the number; obtain a total score table containing all types of garbage and scores based on big data statistics, and obtain the score corresponding to each type of garbage from the total score table; Step 4.3: Multiply the score and quantity to get the score of each type of garbage, and add up the scores of all types of garbage to get the total score S. total ; Step 4.4: Calculate the actual total area A of the target area covered by the acquisition device through the optical parameters of the camera and its angular position parameters with the target area. The calculation process is: Step 4.4.

1. Calculate the scale factor R of the calibration object: Where S is the actual size of the calibration object; P is the pixel size of the calibration object in the photo; Step 4.4.2, calculate the actual size corresponding to the pixel size in the photo; for the plane perpendicular to the camera, the scale factor between the actual size and the pixel size is directly obtained according to the calculation formula of the scale factor of the calibration object. If there is a shooting angle, the influence of the angle needs to be considered; Step 4.4.3, considering the shooting distance and focal length, use the principle of similar triangles to calculate the effect of the distance from the camera to the target area on the size. The calculation formula is as follows: Among them, W actual is the actual width of the target area in the photo; H actual is the actual height of the target area in the photo; W is the width of the target area in the photo; H is the actual height of the target area in the photo; d is the distance from the camera to the target area; f is the focal length of the camera; Step 4.4.4: When the multiple shooting angles are not vertical, consider the effect of the shooting angle on the actual size; assuming the shooting angle is θ, the projection length of the target area will become shorter; the calculation formula is as follows: W adjusted =W actual ×cos(θ); H adjusted =H actual ×cos(θ); Among them, W actual H is the adjusted width of the target area in the photo; actual Adjust the height of the target area in the photo; Step 4.4.5: Calculate the actual area of ​​the target area using the following formula: Step 4.5: Calculate the cleaning index CI based on the total score and the actual area of ​​the target area. The formula is: Among them, α is a tuning parameter used to control the sensitivity of the cleaning index; Step 4.6, set the evaluation standard for sanitation work; when the cleanliness index is greater than 0.8, the sanitation work is judged to be qualified, indicating that the cleanliness has reached an excellent level; when 0.5<cleanliness index≤0.8, the sanitation work is judged to need improvement, indicating that the cleanliness is moderate, but there is still room for improvement; when the cleanliness index≤0.5, the sanitation work is judged to be unqualified, indicating that the cleanliness is poor and improvement measures need to be taken immediately; Step 4.

7. Regularly calculate and record the cleanliness index to evaluate the performance of sanitation work; regularly calculate and record the cleanliness index to form a systematic sanitation work evaluation report; the specific process is as follows: extract image frames from the camera video stream regularly every day or week, calculate the cleanliness index, record the cleanliness index calculated each time, and summarize it to generate time series data; by comparing the changes in the cleanliness index in different time periods, evaluate the effectiveness and continuous improvement of sanitation work, generate a cleanliness report, and put forward corresponding improvement suggestions.

Images