Deep learning-based abandoned mask detection method

Abstract

The present application relates to the technical field of environmental protection, specifically to a kind of abandoned mask detection method based on deep learning, comprising the following steps: S1. abandoned mask training sample image set is constructed;S2. the image feature data of training sample image set is extracted, and the image feature data set of training sample is obtained;The image feature data includes mask contour feature data and mask color enhancement feature data;S3. abandoned mask detection model is constructed and trained and verified;S4. the abandoned mask of the image to be detected is detected using the verified abandoned mask detection model.The present application increases mask contour feature data and mask color enhancement feature data as data source, combined with R, G, B three channels, improves the recognition accuracy of abandoned mask.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of environmental protection, in particular to a discarded mask detection method based on deep learning, which realizes accurate identification of discarded masks relying on visual analysis technology and reduces the pollution of discarded masks to the environment. BACKGROUND

[0002] Respiratory droplets are one of the main transmission routes of viruses, and wearing a mask in public places where social distancing is difficult to maintain can effectively reduce the spread of viruses. Wearing a mask can filter aerosols carrying virus particles and play a key role in suppressing the spread of viruses, but it also brings new problems. In the absence of safety awareness, used masks are discarded at will, becoming a new safety hazard.

[0003] Because viruses can survive for four to seven days in humid conditions, used masks can become a new source of infection. Discarded masks not only pose a threat to human safety, but also directly affect marine life, birds, fisheries, aquaculture, and may trigger larger-scale subsequent disasters. Therefore, it is of great significance to properly handle discarded masks in a timely manner, and timely detection of masks is a prerequisite for timely disposal.

[0004] Real-time detection and alarm of discarded masks by video, camera and other devices installed in public areas is a method for timely detection and rapid disposal of discarded masks, such as the waste mask collection box of the utility model patent with the publication number CN215477456U. However, discarded masks not only exhibit different forms on the ground, but also are easily confused with various ground objects such as plastic bags, paper sheets, stones and bricks. The existing video, camera and other devices in public areas usually only use R, G and B channels to identify them, and due to the limited available information, the accuracy is low.

[0005] Therefore, it is urgent to develop a method that can accurately identify discarded masks. SUMMARY

[0006] The purpose of the present application is to provide a discarded mask detection method based on deep learning to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides the following technical solution: a discarded mask detection method based on deep learning, comprising the following steps:

[0008] S1. Constructing a training sample image set of discarded masks;

[0009] S2. Extracting image feature data of the training sample image set to obtain an image feature data set of the training sample; the image feature data includes mask contour feature data and mask color enhancement feature data;

[0010] S3. Constructing the abandoned mask detection model and training and verifying it;

[0011] S3.1 training the abandoned mask detection model by using part of the image feature data set, obtaining the trained abandoned mask detection model;

[0012] S3.2 verifying the trained abandoned mask detection model by using another part of the image sample data set, and finally obtaining the verified abandoned mask detection model.

[0013] S4. Detecting the abandoned mask of the to-be-detected image by using the verified abandoned mask detection model.

[0014] Further, the construction method of step S1 specifically comprises:

[0015] S1.1. Obtain a plurality of groups of training sample images containing abandoned masks;

[0016] S1.2. Label the abandoned masks in the above training sample images respectively, and obtain a training sample image set of abandoned masks;

[0017] Further, the obtaining method of S1.1 is specifically: using cameras with different heights and angles to shoot abandoned masks in different scenes.

[0018] Further, the mask contour feature data in S2 includes one or more of mask large-area crease feature data, ear line feature data on both sides of the mask, and regular linear structure feature data of the mask.

[0019] Further, the extraction method of the mask contour feature data in S2 is the Canny edge detection algorithm.

[0020] Further, the mask color enhancement feature data in S2 includes the pixel values of the R channel, the G channel, the B channel and the N channel of each pixel in the training sample image, and the N channel is the weight of the RGB channel information.

[0021] Further, when the mask color in the training sample image is blue, the N channel is defined as:

[0022]

[0023] Wherein, r xy g xy b xy respectively represent the pixel values of the R, G and B channels of the xth row and yth column, and N xy represents the pixel value of the N channel of the xth row and yth column.

[0024] Further, when the mask color in the training sample image is white, the N channel is defined as:

[0025]

[0026] wherein r xy g xy b xy respectively represent the pixel values of the R, G and B channels of the xth row and yth column, N xy represents the pixel value of the N channel of the xth row and yth column, and m is the average value of the three channels.

[0027] Further, when the mask color in the training sample image is black, the N channel is defined as:

[0028]

[0029] wherein r xy g xy b xy respectively represent the pixel values of the R, G and B channels of the xth row and yth column, N xy represents the pixel value of the N channel of the xth row and yth column.

[0030] Further, when the mask color in the training sample image is red, the N channel is defined as:

[0031]

[0032] wherein r xy g xy b xy respectively represent the pixel values of the R, G and B channels of the xth row and yth column, N xy represents the pixel value of the N channel of the xth row and yth column.

[0033] Further, when the mask color in the training sample image is pink, the N channel is defined as:

[0034]

[0035] wherein r xy g xy b xy respectively represent the pixel values of the R, G and B channels of the xth row and yth column, N xy represents the pixel value of the N channel of the xth row and yth column.

[0036] Further, the abandoned mask detection model in S4 is a YOLOv5s model.

[0037] Compared with the prior art, the present application has the following beneficial effects:

[0038] This invention discloses a deep learning-based method for detecting abandoned masks. Based on the mask's contour features and color enhancement features, this application uses the mask's contour feature data after edge detection and the mask's color enhancement feature data with added N channels as data sources. These data are combined with the R, G, and B channels and incorporated into deep learning. An improved YOLOv5s model is used to learn and mine multi-type information. Compared to mask recognition methods that only use the R, G, and B channels of color images as data sources, the detection method disclosed in this application can accurately detect abandoned masks. The fusion of multi-type information improves the recognition accuracy of abandoned masks. Attached Figure Description

[0039] Figure 1 This refers to a subset of images from the training sample image set.

[0040] Figure 2 This is an image from the training sample image set;

[0041] Figure 3 for Figure 2 Image showing the extraction results of the contour feature data of a Chinese-made face mask;

[0042] Figure 4 for Figure 2 N-channel feature map;

[0043] Figure 5 Images showing partial test results for discarded masks in different scenarios;

[0044] Figure 6 The existing technology uses RGB three-channel data as the abandoned mask detection model to show the detection results of abandoned masks in different scenarios;

[0045] Figure 7 This application uses RGB three-channel data, mask outline feature data, and N-channel data as the abandoned mask detection model to show the detection results of abandoned masks in different scenarios. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] This invention provides a technical solution: a deep learning-based method for detecting abandoned face masks, comprising the following steps:

[0048] S1. Construct a training sample image set of abandoned face masks, specifically including:

[0049] S1.1. Obtain multiple sets of training sample images containing discarded masks, specifically: use cameras at different heights and angles to photograph discarded masks in different scenes;

[0050] S1.2. Label the abandoned masks in the above training sample images to obtain a training sample image set of abandoned masks;

[0051] S2. Extract image feature data from the training sample image set to obtain the image feature dataset of the training samples;

[0052] Image feature data includes mask outline feature data and mask color enhancement feature data. Mask outline feature data includes one or more of the following: large area crease feature data, ear line feature data on both sides of the mask, and regular linear structure feature data of the mask. The mask outline feature data is extracted using the Canny edge detection algorithm.

[0053] The mask color enhancement feature data includes the pixel values ​​of the R, G, B and N channels of each pixel in the training sample image, where the N channel is the weight of the RGB channel information;

[0054] When the mask color in the training sample image is blue, the N-channel is defined as:

[0055]

[0056] Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column.

[0057] S3. Construct a model for detecting abandoned masks and train and validate it;

[0058] S3.1 uses a portion of the image feature dataset to train the abandoned mask detection model, and iterates repeatedly to obtain the trained abandoned mask detection model;

[0059] S3.2 uses another part of the image feature dataset to validate the trained abandoned mask detection model, and finally obtains the validated abandoned mask detection model.

[0060] S4. The validated abandoned mask detection model is used to detect abandoned masks in the image to be detected. The abandoned mask detection model is the YOLOv5s model.

[0061] Please see Figure 1In this embodiment, blue medical masks were used as the research object. Photos of abandoned masks in different environments were taken from different heights and angles. Abandoned masks in different scenarios were manually marked to generate a training sample image set of abandoned masks.

[0062] Different objects in an image possess different colors and spatial geometric features, and face masks primarily rely on color and shape information to distinguish them from other objects. Enhancing the edges and color enhancement features of objects in an image is more beneficial for face mask recognition. Therefore, this application creatively incorporates face mask contour feature data and face mask color enhancement feature data when extracting image feature data from the training sample image set.

[0063] Regarding the contour features of face masks, the applicant, through analysis of a large number of face mask structures, especially discarded and abandoned masks, found that compared with most other ground features, face masks have the following prominent characteristics: 1) Most ordinary disposable face masks have creases; 2) Face masks have ear loops on both sides, belonging to a combination of line and surface structure; 3) Face masks have regular linear structures. Different types of face masks possess one or more of these features. These features can generally be highlighted using edge detection methods. Therefore, extracting edge contour features of face masks before recognition can effectively improve detection accuracy. Edge contours are a collection of gray-level abrupt changes in an image, corresponding to high-frequency information in the frequency domain, containing a large amount of useful information in the image. Canny is a classic edge detection algorithm that can extract useful structural information from different visual objects and greatly reduce the amount of data to be processed. This application uses the Canny algorithm to obtain the edge features of different objects in the image. Gaussian filtering is used to smooth the image, and the magnitude and direction of the gray-level gradient are calculated through a window. Based on the distribution of the gray-level histogram, non-maximum suppression of the gradient is applied, and a double thresholding method is used for edge detection and connection of candidate edge points. Please refer to [link to relevant documentation]. Figure 2 The edges and contours of different objects are clearly defined.

[0064] For enhancing the color of face masks, a single, striking hue is another important characteristic of common masks. Medical disposable masks are predominantly blue, with other colors including white; N95 masks are predominantly white, with a small number in black. These hues differ somewhat from most natural features and can serve as important auxiliary information for mask identification. Therefore, to improve mask recognition accuracy, an N-channel enhancement technique (color enhancement technology) from image processing is used to enhance the difference between the mask's hue and other natural features. Adding an N-channel allows for optimal adjustment of the input image's saturation, brightness, and skin tone. Taking the most common blue medical mask as an example, each pixel in the image has R, G, and B channels representing red, green, and blue respectively, all integers within the range [0, 255]. To highlight the blue channel information, a new N-channel is added, which is the weight of the RGB channel data. (See [link to relevant documentation]). Figure 3 As shown, the B channel has a higher weight than other colors, thus highlighting the blue image, which is helpful in distinguishing masks from other objects.

[0065] In training the abandoned mask detection model, YOLO is categorized as a Convolutional Neural Network (CNN) model, one of the deep learning algorithms based on Python. Currently, there are five main versions, and YOLOv5s has been widely used in object detection due to its good flexibility and high efficiency. Therefore, YOLOv5s was chosen as the base network for abandoned mask detection. This application improves the input module by using the R, G, and B channel data, as well as the mask contour feature data and N channel data, from the input to the YOLOv5s model. Please refer to [link to relevant documentation]. Figure 4 As shown, the improved YOLOv5s model consists of four parts: the input module, the backbone module, the neck module, and the head module. The main function of the input module is data input and processing. Since the original image size is not fixed, considering computational efficiency and the effective information of the image, the original image is resampled to 640*640 for feedback to the detection network. To increase sample diversity, the image is flipped, scaled, and its color gamut is changed to expand the data volume, which also enhances the network's feature learning ability. The backbone module mainly includes a focusing module, a feature extraction network, and corresponding activation functions. The focusing module slices the image before it enters the backbone network. This reduces the computational load of the model to some extent without losing information and preserves more complete image sampling information for subsequent feature extraction. To better utilize the features extracted by the backbone, the neck module is located between the backbone and the head module. The head module is used for prediction based on the extracted features.

[0066] Comparative test

[0067] Two hundred images of discarded face masks distributed across different scenes were collected, with 150 used for training and 50 for validation. The experiments were conducted on a computer with a 3.6GHz Intel i9-9900K CPU and an NVIDIA RTX 2080Ti GPU, using Python within the PyTork framework. Four images were fed into the discarded face mask detection model at each step, with a learning rate of 0.0001 and a learning iteration count of 2000. The loss value gradually decreased with increasing learning iterations. When the number of iterations reached 1000, the learning process stabilized, and the loss value stabilized at 0.05. After validation with the 50 discarded face mask images, face mask recognition could be performed. Some of the recognition results can be found in [link to relevant documentation]. Figure 5 .

[0068] like Figure 6 and 7 A comparison was made between using existing RGB three-channel data as input to the YOLOv5s model and the method of this application, which integrates mask contour features and N-channel data into the RGB three-channel data as input to the YOLOv5s model, to identify masks in the same set of images. The results showed that the detection method disclosed in this application outperformed the RGB three-channel detection method in identifying abandoned masks. In different scenarios such as grass, parking lots, bushes, and asphalt roads, the average accuracy rate of the detection method disclosed in this application for identifying abandoned masks was 94.3%, higher than the average of 88.8% for the RGB three-channel detection method. Furthermore, this method can also accurately detect partially obscured masks and folded masks.

[0069] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based method for detecting abandoned face masks, characterized in that: Includes the following steps: S1. Construct a training sample image set of abandoned face masks; S2. Extract image feature data from the training sample image set to obtain the image feature dataset of the training samples; the image feature data includes mask outline feature data and mask color enhancement feature data; S3. Construct a model for detecting abandoned masks and train and validate it; S3.1 uses partial data from image samples and feature datasets to train the abandoned mask detection model, and iterates cyclically to obtain the trained abandoned mask detection model; S3.2 uses another part of the image sample dataset to validate the trained abandoned mask detection model, and finally obtains the validated abandoned mask detection model; S4. The validated abandoned mask detection model is used to detect abandoned masks in the images to be detected; The mask is a single color; the mask color enhancement feature data in S2 includes the pixel values ​​of the R channel, G channel, B channel and N channel of each pixel in the training sample image, and the N channel is the feature data after color enhancement; When the mask color in the training sample image is blue, the N channels are defined as follows: Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column; When the mask color in the training sample image is white, the N channel is defined as: Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column, where m is the average value of the three channels; When the mask color in the training sample image is black, the N channel is defined as: Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column; When the mask color in the training sample image is red, the N channel is defined as: Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column; When the mask color in the training sample image is pink, the N channel is defined as: Where, r xy g xy b xy N represents the pixel values ​​of the R, G, and B channels in the x-th row and y-th column, respectively. xy This represents the pixel value of the N channel in the x-th row and y-th column.

2. The method for detecting abandoned masks based on deep learning according to claim 1, characterized in that: The construction method of step S1 specifically includes: S1.

1. Obtain multiple sets of training sample images containing discarded face masks; S1.

2. Label the abandoned masks in the above training sample images to obtain a training sample image set of abandoned masks.

3. The method for detecting abandoned masks based on deep learning according to claim 2, characterized in that: The method for obtaining S1.1 is as follows: using cameras at different heights and angles to photograph abandoned masks in different scenarios.

4. The method for detecting abandoned masks based on deep learning according to claim 1, characterized in that: The mask contour feature data in S2 includes one or more of the following: large area crease feature data of the mask, ear line feature data on both sides of the mask, and regular linear structure feature data of the mask.

5. The method for detecting abandoned masks based on deep learning according to claim 4, characterized in that: The method for extracting the mask contour feature data in S2 is the Canny edge detection algorithm.

6. The method for detecting abandoned masks based on deep learning according to claim 1, characterized in that: The abandoned mask detection model in S4 is the YOLOv5s model.

Images