Mask wearing recognition method and device, electronic equipment and storage medium
By combining multi-frame image analysis and historical frames of video data, the problem of misjudgment and missed judgment in mask wearing recognition in the existing technology is solved, and the recognition accuracy is improved, especially in complex angles and occlusion conditions, it can accurately judge the mask wearing status.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LENS SYST INTEGRATION CO LTD
- Filing Date
- 2023-04-04
- Publication Date
- 2026-07-14
Smart Images
Figure CN116403240B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image recognition technology, and more specifically, to a mask wearing recognition method, device, electronic device, and storage medium. Background Technology
[0002] Currently, the mask-wearing recognition technologies that rely on face detection or direct object detection algorithms mostly analyze and judge single-frame images, which leads to misjudgments or omissions, resulting in low accuracy in mask-wearing recognition.
[0003] There are currently no effective solutions to the aforementioned problems in the relevant technologies. Summary of the Invention
[0004] This application provides a mask wearing recognition method, device, electronic device, and storage medium to solve the technical problems of false or missed recognition and low recognition accuracy in related technologies for mask wearing recognition.
[0005] According to one aspect of the embodiments of this application, a mask wearing recognition method is provided, comprising: acquiring video data; performing head target detection on a target object in the current frame image of the video data and cropping a head region image; inputting the head region image into a mask wearing recognition model to obtain a first model discrimination result of the head region image, wherein the first model discrimination result is a mask wearing category derived by the model for the current frame head region image of the target object; acquiring multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model for the historical frame head region images of the target object; and determining the final mask wearing category of the target object based on the first model discrimination result and the second model discrimination result.
[0006] Further, inputting the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image includes: inputting the head region image into the mask wearing recognition model, obtaining the confidence scores of multiple mask wearing categories corresponding to the head region image; obtaining the first mask wearing category with the highest confidence score, and determining whether the highest confidence score is greater than a preset threshold; if the highest confidence score is greater than the preset threshold, then the first mask wearing category is taken as the first model discrimination result; if the highest confidence score is less than or equal to the preset threshold, then the first model discrimination result is determined to be that the wearing status is uncertain.
[0007] Further, determining the final mask-wearing category of the target object based on the first model discrimination result and the second model discrimination result includes: acquiring multiple second model discrimination results and the second mask-wearing category that appears most frequently in the first model discrimination result; determining whether the proportion of the second mask-wearing category is greater than a preset proportion; if the proportion of the second mask-wearing category is greater than the preset proportion, then the second mask-wearing category is used as the intermediate logical result of the head region image of the current frame; and determining the final mask-wearing category of the target object based on the intermediate logical result.
[0008] Further, determining the final mask-wearing category of the target object based on the intermediate logic result includes: determining whether the intermediate logic result indicates an uncertain wearing status; if the intermediate logic result indicates an uncertain wearing status, then using the final wearing category of the target object's previous frame head region image; if the intermediate logic result does not indicate an uncertain wearing status, then updating the final mask-wearing category of the target object to the intermediate logic result.
[0009] Furthermore, after determining the final mask-wearing category of the target object based on the first model discrimination result and the second model discrimination result, the method further includes: if the final wearing category is determined to be not wearing a mask, then capturing the head area image and recording the monitoring address, occurrence time, duration of not wearing a mask, and the person ID of the target object corresponding to the head area image.
[0010] Furthermore, before inputting the head region image into the mask-wearing recognition model to obtain the first model discrimination result of the head region image, the method further includes: acquiring mask-wearing samples; setting sample labels for the mask-wearing samples, wherein the sample labels include properly worn, improperly worn, not worn, and wearing status uncertain; and training the initial model using the mask-wearing samples and the following loss function to obtain the mask-wearing recognition model: Where m is the number of mask-wearing samples, n is the number of multi-label categories, and y i Let be the label value corresponding to the i-th label, where the label value is either 0 or 1. Let P be the probability value predicted by the mask-wearing recognition model for each label category, where the probability value ranges from [0, 1]. It is the sum of the cross-entropy of all labels from i=1 to i=n.
[0011] Furthermore, performing head target detection on the target object in the current frame image of the video data and extracting the head region image includes: obtaining the head region obtained after head target detection; expanding the region range of the head region outward and extracting the head region image from the current frame image.
[0012] According to another aspect of the embodiments of this application, a mask wearing recognition device is also provided, comprising: an acquisition module for acquiring video data; a detection module for detecting a head target in a current frame image of the video data and cropping a head region image; a first discrimination module for inputting the head region image into a mask wearing recognition model to obtain a first model discrimination result of the head region image, wherein the first model discrimination result is a mask wearing category derived by the model for the current frame head region image of the target object; the acquisition module is further configured to acquire multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model for the historical frame head region images of the target object; and a second discrimination module for determining the final mask wearing category of the target object based on the first model discrimination result and the second model discrimination result.
[0013] Further, the first discrimination module includes: a first discrimination unit, configured to input the head region image into the mask wearing recognition model, obtain the confidence scores of multiple mask wearing categories corresponding to the head region image; obtain the first mask wearing category with the highest confidence score, and determine whether the highest confidence score is greater than a preset threshold; if the highest confidence score is greater than the preset threshold, then the first mask wearing category is used as the first model discrimination result; if the highest confidence score is less than or equal to the preset threshold, then the first model discrimination result is determined to be that the wearing status is uncertain.
[0014] Furthermore, the second discrimination module includes a second discrimination unit, used to acquire multiple second model discrimination results and the second mask wearing category that appears most frequently in the first model discrimination results; determine whether the proportion of the second mask wearing category is greater than a preset proportion; if the proportion of the second mask wearing category is greater than the preset proportion, then the second mask wearing category is used as the intermediate logical result of the head region image of the current frame; and determine the final mask wearing category of the target object based on the intermediate logical result.
[0015] Furthermore, the second discrimination module also includes a third discrimination unit, used to determine whether the intermediate logic result is an uncertain wearing status; if the intermediate logic result is an uncertain wearing status, the final wearing category of the previous frame head region image of the target object is used; if the intermediate logic result is not an uncertain wearing status, the final wearing category of the mask worn by the target object is updated to the intermediate logic result.
[0016] Furthermore, the mask wearing recognition device also includes a recording module, which is used to capture the head area image if the final wearing category is determined to be not wearing a mask, and record the monitoring address, occurrence time, duration of not wearing a mask, and the person ID of the target object corresponding to the head area image.
[0017] Furthermore, the mask-wearing recognition device also includes a model training module for acquiring mask-wearing samples; setting sample labels for the mask-wearing samples, wherein the sample labels include properly worn, improperly worn, not worn, and wearing status uncertain; and training an initial model using the mask-wearing samples and the following loss function to obtain a mask-wearing recognition model: Where m is the number of mask-wearing samples, n is the number of multi-label categories, and y i Let be the label value corresponding to the i-th label, where the label value is either 0 or 1. Let P be the probability value predicted by the mask-wearing recognition model for each label category, where the probability value ranges from [0, 1]. It is the sum of the cross-entropy of all labels from i=1 to i=n.
[0018] Furthermore, the detection module includes a detection unit, used to perform human head target detection on the target object in the current frame image of the video data, and to extract the head region image, including: obtaining the head region obtained after head target detection; expanding the region range of the head region outward, and extracting the head region image from the current frame image.
[0019] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above steps when the program is run.
[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the programs stored in the memory.
[0021] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above-described method.
[0022] This application first uses a mask-wearing recognition model to identify the head region image of the current frame, obtaining the model's identification result. Then, it combines the model's identification results from multiple historical frames of the target person for comprehensive analysis. By combining historical states, it comprehensively determines the final mask-wearing status of the current person. If the person is wearing a mask correctly but has their head down or face turned to the side, making it impossible to identify the face, it does not fail to perform the mask-wearing recognition task or forcibly determine that no mask is being worn, as is the case in related technologies. Instead, it combines historical wearing status analysis to determine the mask-wearing status, reducing the problem of misjudgment or missed judgment and improving the accuracy of mask-wearing recognition. Attached Figure Description
[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0024] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application;
[0025] Figure 2 This is a flowchart of a mask wearing recognition method according to an embodiment of this application;
[0026] Figure 3 This is a schematic diagram of the process for updating the intermediate logic result of the current frame in an embodiment of this application;
[0027] Figure 4 This is a flowchart illustrating the historical result update process of an embodiment of this application;
[0028] Figure 5 This is a structural block diagram of a mask wearing recognition device according to an embodiment of this application. Detailed Implementation
[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0031] Example 1
[0032] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile phone, computer, tablet, or similar computing device. Taking running on a computer as an example, Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application. Figure 1 As shown, a computer may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the computer may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer described above. For example, the computer may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0033] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a mask-wearing recognition method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a computer's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.
[0035] This embodiment provides a mask wearing recognition method. Figure 2 This is a flowchart of a mask wearing recognition method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:
[0036] Step S10: Acquire video data;
[0037] Step S20: Detect human head targets in the target object in the current frame image of the video data, and extract the head region image;
[0038] The current frame image contains information on multiple individuals. A unique ID (Identity Document, account number, code, etc.) is assigned to each detected individual's head information for tracking. A head tracking model such as Deepsort can be used, combined with feature similarity calculations from a deep learning model for cascaded matching. Appropriate parameters are set based on the tracking difficulty and matching time required in the actual project. Individual head detection can employ object detection models. This embodiment uses the YOLOx model for training and inference. The collected sample images come from external open-source individual head data, combined with internally collected real-world images, broadening the applicability. Other object detection models, such as SSD and Faster-RCNN, can also be used.
[0039] This embodiment performs head detection alone, without simultaneously performing head target detection and mask wearing category recognition through a target detection model. In contrast, performing head detection alone is not constrained by the joint loss function of classification and location boxes, which can improve detection performance and generalization performance.
[0040] Step S30: Input the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object.
[0041] Step S40: Obtain multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model from the historical frame head region image of the target object;
[0042] Step S50: Determine the final wearing category of the mask worn by the target object based on the discrimination results of the first model and the discrimination results of the second model.
[0043] By inputting the head region image of the target object into the mask wearing recognition model, the mask wearing category of the current frame can be obtained by the mask wearing recognition model based on the current frame head region image of the target object.
[0044] This embodiment uses ResNet50 as the baseline model. The reason for not choosing a smaller, lightweight model is that the head region in the monitoring scenario is often blurry and small, limiting the recognition capability of a lightweight model. The mask-wearing classification results from multiple historical frames of the target object's head region images, determined by the mask-wearing recognition model, are then obtained. For example, N second-model classification results are obtained for N historical images. A comprehensive analysis of the multi-frame model classification results, based on the first-model classification results and the N second-model classification results, is then performed to determine the final mask-wearing classification of the target object. This final classification is the mask-wearing classification obtained after multiple assessments. This embodiment analyzes and classifies based on multiple frames of images, reducing the misjudgment problem inherent in single-frame image analysis and improving the accuracy of mask-wearing recognition.
[0045] Through the above steps, the head region image of the current frame is first judged by the mask wearing recognition model to obtain the model's judgment result. Then, the model's judgment results from multiple historical frames of the target person are comprehensively analyzed. The final mask wearing status is judged by combining the historical status. If the person is wearing a mask properly but cannot be recognized due to looking down or turning their face to the side, the mask wearing recognition task is not skipped as in related technologies, or a forced judgment of not wearing a mask is made. Instead, the mask wearing status is judged by combining the historical wearing status, which reduces the problem of misjudgment or missed judgment and improves the accuracy of mask wearing recognition.
[0046] In another embodiment of this example, detecting human head targets in the target object in the current frame image of the video data and cropping the head region image includes:
[0047] S21, Obtain the head region obtained after head target detection;
[0048] S22, expand the area of the head region outward, and extract the head region image from the current frame image.
[0049] In this embodiment, it should be noted that the head region in the target detection annotation is only the smallest rectangle including the head. However, when used as a training dataset for the mask wearing recognition model or for model inference, the head region needs to be expanded and cropped. In one example, the area of the head region that is expanded outward can be expanded by 10% in the directions of up, down, left, and right. More specifically, the position of the head region can be obtained by taking the x-coordinate of the center point of the head region, the y-coordinate, the width w, and the height h. Taking the center point as the reference coordinate, the width (1+a)w and the height (1+a)h are taken, where a is the percentage of expansion that can be taken according to the actual situation, for example, 20%. This ratio can ensure that the cropped head region includes the surrounding situation of the face and mask wearing.
[0050] In one embodiment of this example, inputting the head region image into a mask-wearing recognition model to obtain a first model discrimination result for the head region image includes:
[0051] S31, input the head region image into the mask wearing recognition model, and obtain the confidence scores of multiple mask wearing categories corresponding to the head region image respectively;
[0052] S32, obtain the first mask wearing category with the highest confidence level, and determine whether the highest confidence level is greater than a preset threshold;
[0053] S33, if the maximum confidence level is greater than the preset threshold, the first mask wearing category is taken as the first model discrimination result; if the maximum confidence level is less than or equal to the preset threshold, the first model discrimination result is determined to be that the wearing status is uncertain.
[0054] Mask wearing categories include wearing correctly, not wearing correctly, not wearing, and wearing status uncertain. The category of uncertain wearing status includes situations such as the face being turned away from the camera, the head being lowered, the face being turned to the side, or other obstructions covering the face, as well as situations where it is impossible to determine the person's mask-wearing status. The category of not wearing correctly includes situations such as wearing a mask but not covering the nose.
[0055] In one example, refer to Figure 3The head region image of the target object in the current frame is processed by the mask-wearing recognition model to obtain the confidence scores for the four categories mentioned above: P1 (confidence score of "properly worn"), P2 (confidence score of "not properly worn"), P3 (confidence score of "not worn"), and P4 (confidence score of "uncertain wearing status"). The maximum value of the four confidence scores, max(P1, P2, P3, P4), is taken to obtain the first mask-wearing category with the highest confidence score. In this embodiment, the first mask-wearing category with the highest confidence score is denoted as ClassX. ClassX can be any of the four categories mentioned above, and its confidence score is set to Pn. The maximum confidence score P is then determined. If n is greater than a preset threshold, the preset threshold can be set according to specific circumstances (e.g., 50%). If Pn is greater than 50%, then ClassX is used as the first model's discrimination result. If Pn is less than or equal to 50%, it means that the probability of the first mask wearing category ClassX with the highest confidence is still low, and the mask wearing category cannot be accurately determined. Therefore, the first model's discrimination result is determined to be the category of uncertain wearing status. In this embodiment, the situation where the mask wearing category cannot be accurately determined is classified as uncertain wearing status. This avoids the unreasonable allocation of the three categories of "properly worn," "not properly worn," or "not worn" when the mask wearing status cannot be accurately determined, which can reduce misjudgment.
[0056] In one embodiment of this example, determining the final mask-wearing category of the target object based on the first model discrimination result and the second model discrimination result includes:
[0057] S51, obtain multiple second model discrimination results and the second mask wearing category that appears most frequently in the first model discrimination results;
[0058] S52, determine whether the proportion of the second mask wearing category is greater than the preset proportion;
[0059] S53, if the proportion of the second mask wearing category is greater than the preset proportion, then the third mask wearing category is used as the intermediate logical result of the head region image of the current frame.
[0060] S54, determine the final wearing category of the mask for the target object based on the intermediate logic result.
[0061] After the head region image of the target object in the current frame is processed by the mask wearing recognition model to obtain the first model discrimination result, the first model discrimination result is stored in the model discrimination result queue of the target object ID. The last preset number (e.g., 10 frames) of model discrimination results in the model discrimination result queue are obtained, such as obtaining the model discrimination results (second model discrimination results) corresponding to the head region images of the target object in 9 historical frames and the first model discrimination result corresponding to the head region image of the current frame. The second mask wearing category that appears most frequently among the multiple model discrimination results is calculated. In this embodiment, the second mask wearing category that appears most frequently is denoted as ClassY. The second mask wearing category ClassY is then determined. The system checks whether the proportion of Class Y in multiple second model discrimination results and first model discrimination results is greater than a preset proportion. The preset proportion can be set according to specific circumstances (e.g., 50%). If the proportion of Class Y in the 10-frame model discrimination results is greater than 50%, then the class Class Y that appears most frequently is used as the intermediate logical result of the current frame head region image. If the proportion of the second mask wearing category is less than or equal to the preset proportion, then the intermediate logical result corresponding to the previous historical frame head region image of the target object is used as the intermediate logical result of the current frame head region image. Then, the final mask wearing category of the target object is determined based on the intermediate logical result of the current frame head region image.
[0062] It should be noted that the length of the historical image frames can be adjusted according to the actual situation. In one example, such as... Figure 3 The flowchart shown illustrates the intermediate logic result update process for the current frame's head region image. The head region image of the target object ID in the current frame image (k) is output by a classification model (the mask-wearing recognition model in this embodiment). The class with the highest confidence among the four labels, ClassX, is selected. It is determined whether this highest confidence is greater than 0.5. If the highest confidence is greater than 0.5, the current frame model's discrimination result is the class with the highest confidence, ClassX. If the highest confidence is less than or equal to 0.5, the current frame model's discrimination result is "uncertain wearing status." The current frame model's discrimination result is then added to the target object ID's model discrimination result queue. The results of the previous 10 frames of the current frame image are taken, and the class with the most categories, ClassY, is calculated. It is determined whether the proportion of ClassY is greater than 50%. If the proportion of ClassY is greater than 50%, the intermediate logic result of the current frame is ClassY. If the proportion of ClassY is less than or equal to 50%, the intermediate logic result of the current frame is the intermediate logic result of the previous frame (k-1) for the target object ID.
[0063] In one embodiment of this example, determining the final mask-wearing category of the target object based on the intermediate logic result includes:
[0064] S541, determine whether the intermediate logic result is an uncertain wearing status;
[0065] S542, if the intermediate logic result is that the wearing status is uncertain, then the final wearing category of the previous frame head region image of the target object shall be used.
[0066] S543, if the intermediate logic result is not "wearing status uncertain", then update the final wearing category of the mask worn by the target object to the intermediate logic result.
[0067] In this embodiment, the final wearing category is recorded as the historical result, and the initial historical result of the target object is set to an uncertain wearing state. That is, the historical result is updated according to the intermediate logical result of the current frame head region image, and the updated historical result is used as the final wearing category. Specifically, it is determined whether the intermediate logical result of the current frame head region image is an uncertain wearing state. If the intermediate logical result is an uncertain wearing state, the final wearing category of the previous frame head region image of the target object is not updated, that is, the final wearing category of the current frame head region image is the final wearing category of the previous frame head region image; if the intermediate logical result is not an uncertain wearing state (that is, it is properly worn, improperly worn, or not worn), the intermediate logical result is used as the final wearing category of the current frame head region image, that is, the historical result is updated. If the historical result of the previous frame's head region image is "Standardized Wearing," and the current frame's intermediate logical result is "Not Wearing," then the historical result for this ID will be updated to "Not Wearing." If the current frame's intermediate logical result is "Not Standardized Wearing," then the historical result for this ID will be updated to "Not Standardized Wearing." If the current frame's logical result is "Standardized Wearing," then the historical result for this ID will remain "Standardized Wearing." If the current frame's logical result is "Wearing Status Uncertain," then the historical result for this ID will not be updated and will continue to be "Standardized Wearing." In one example, refer to... Figure 4 The ID's historical result is initialized as "uncertain wearing status". The intermediate logical result of the current frame's head region image is obtained, and it is determined whether the current frame's intermediate logical result is "uncertain wearing status". If the intermediate logical result is "uncertain wearing status", the historical result of this ID remains unchanged; if the intermediate logical result is not "uncertain wearing status", the historical result of this ID is updated to the intermediate logical result of the current frame's head region image. In this embodiment, when the field of vision and angle are poor, the pedestrian's mask-wearing status remains uncertain, continuing to be the previous state. When a more certain state change occurs, the corresponding pedestrian's mask-wearing status changes accordingly.
[0068] For real-time analysis, the head region of each pedestrian is marked in the current frame image, and the corresponding person ID, the current frame logical result, and the person's historical results are displayed. The historical results may differ from the current frame logical results. For example, the mask-wearing status in the current frame image may be "uncertain" due to obstruction or blurring, but the historical results indicate that the person was previously "wearing the mask correctly." Combining multi-frame judgment and status continuation methods reduces the false alarm rate of monitoring judgment to less than 1%, greatly improving accuracy.
[0069] In one embodiment of this example, after determining the final mask-wearing category of the target object based on the first model discrimination result and the second model discrimination result, the method further includes: if the final wearing category is determined to be not wearing a mask, then capturing the head area image and recording the monitoring address, occurrence time, duration of not wearing a mask, and the person ID of the target object corresponding to the head area image.
[0070] If the final wearing category is updated and determined to be "not wearing", obtain the target person's ID, capture the head area image of that person's ID, and expand the head area image outward by a preset range, such as expanding it outward by 100%, to facilitate displaying the target person's position in the current frame's monitoring image. Store the captured head area image, which can be recorded by naming the monitoring address, occurrence time, person's ID, and duration of not wearing a mask. For example, the naming method can be "monitoring address-occurrence time (year / month / day / hour / minute / second)-ID:xxx(N)", where N is the duration, and the duration can be accumulated in minutes. For example, if the final wearing category of the person's ID is updated to "standard wearing" within one minute, the captured head area image is retained. When the final wearing category is updated to "not wearing" again, the head area image of the person's ID is captured again, replacing the previous image, and the occurrence time and duration of not wearing a mask are updated, for example, by incrementing the duration N of the previous image by 1.
[0071] In another embodiment of this example, before inputting the head region image into the mask-wearing recognition model to obtain the first model discrimination result of the head region image, the method further includes: acquiring mask-wearing samples; setting sample labels for the mask-wearing samples, wherein the sample labels include properly worn, improperly worn, not worn, and wearing status uncertain; and training an initial model using the mask-wearing samples and the following loss function to obtain a mask-wearing recognition model: Where m is the number of mask-wearing samples, n is the number of multi-label categories, yi is the label value corresponding to the i-th label, the label value is 0 or 1, and P is the probability value predicted by the mask-wearing recognition model for each label category, the probability value is in the range [0, 1]. It is the sum of the cross-entropy of all labels from i=1 to i=n.
[0072] This embodiment uses binary cross entropy (BCE) as the loss function. The initial model is trained using mask-wearing samples and the loss function to obtain a mask-wearing recognition model. i log(P)-(1-y i [log(1-P)] is the binary cross-entropy. In this embodiment, the multi-label categories include four categories: properly worn, improperly worn, not worn, and wearing status uncertain. Therefore, n is 4, and the labels are Y = {y1, y2, ..., y}. n The probability value P can be the confidence level in this embodiment, and the probability value ranges from [0, 1], that is, any value between 0 and 1.
[0073] The recognition model in this embodiment uses a multi-label classification method, where the data label for each mask wearing category is 0 or 1. In the data annotation, each category in this multi-label method is allowed to have at most one category with a confidence level of 1. For example, the label for annotating a head region bounding box that has been properly worn with a mask is [1, 0, 0, 0]. To improve the learning of the classification model, during the training process, images of similar size are randomly cropped from the image and used as negative samples for learning. Their labels are [0, 0, 0, 0]. The intersection-union ratio (IoU) of the randomly cropped image and the labeled head region bounding box is 0, meaning there is no region overlap. Using a multi-label approach allows for more flexible integration with business operations to determine the tightness of mask-wearing identification results. For example, it may be difficult to uniformly label the side profile as either "mask worn correctly" or "wearing status uncertain." When the multi-label classification model results have high confidence on both labels, if the "loose control" approach is chosen, the mask-wearing status can be determined as long as the "mask worn correctly" label is greater than a specified threshold. If the "strict control" approach is chosen, the "wearing status uncertain" status can be determined as "wearing status uncertain" as long as the "wearing status uncertain" label is greater than a threshold, and the "mask worn correctly" status is not determined at this time.
[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0075] Example 2
[0076] This embodiment also provides a mask-wearing recognition device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0077] Figure 5 This is a structural block diagram of a mask wearing recognition device according to an embodiment of this application, such as... Figure 5 As shown, the device includes: an acquisition module 60, a detection module 61, a first discrimination module 62, and a second discrimination module 63, wherein...
[0078] Module 60 is used to acquire video data;
[0079] The detection module 61 is used to detect human head targets in the current frame image of the video data and to extract the head region image.
[0080] The first discrimination module 62 is used to input the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object.
[0081] The acquisition module 60 is further configured to acquire multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model from historical frame head region images of the target object;
[0082] The second discrimination module 63 is used to determine the final wearing category of the mask worn by the target object based on the discrimination results of the first model and the discrimination results of the second model.
[0083] Optionally, the first discrimination module includes: a first discrimination unit, configured to input the head region image into a mask wearing recognition model, obtain the confidence scores of multiple mask wearing categories corresponding to the head region image; obtain the first mask wearing category with the highest confidence score, and determine whether the highest confidence score is greater than a preset threshold; if the highest confidence score is greater than the preset threshold, then the first mask wearing category is used as the first model discrimination result; if the highest confidence score is less than or equal to the preset threshold, then the first model discrimination result is determined to be that the wearing status is uncertain.
[0084] Optionally, the second discrimination module includes a second discrimination unit, used to acquire multiple second model discrimination results and the two mask wearing categories that appear most frequently in the first model discrimination results; determine whether the proportion of the second mask wearing category is greater than a preset proportion; if the proportion of the second mask wearing category is greater than the preset proportion, then the second mask wearing category is used as the intermediate logical result of the head region image of the current frame; and determine the final mask wearing category of the target object based on the intermediate logical result.
[0085] Optionally, the second discrimination module further includes a third discrimination unit, used to determine whether the intermediate logic result is an uncertain wearing status; if the intermediate logic result is an uncertain wearing status, the final wearing category of the previous frame head region image of the target object is used; if the intermediate logic result is not an uncertain wearing status, the final wearing category of the mask worn by the target object is updated to the intermediate logic result.
[0086] Optionally, the mask wearing recognition device further includes a recording module, which, if the final wearing category is determined to be not wearing a mask, captures an image of the head area and records the monitoring address, occurrence time, duration of not wearing a mask, and the personnel ID of the target object corresponding to the head area image.
[0087] Optionally, the mask-wearing recognition device further includes a model training module for acquiring mask-wearing samples; setting sample labels for the mask-wearing samples, wherein the sample labels include properly worn, improperly worn, not worn, and wearing status uncertain; and training an initial model using the mask-wearing samples and the following loss function to obtain a mask-wearing recognition model: Where m is the number of mask-wearing samples, n is the number of multi-label categories, and y i Let be the label value corresponding to the i-th label, where the label value is either 0 or 1. Let P be the probability value predicted by the mask-wearing recognition model for each label category, where the probability value ranges from [0, 1]. It is the sum of the cross-entropy of all labels from i=1 to i=n.
[0088] Optionally, the detection module includes a detection unit for detecting human head targets in the current frame image of the video data and extracting a head region image, including: obtaining the head region obtained after head target detection; expanding the area of the head region outward and extracting the head region image from the current frame image.
[0089] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.
[0090] Example 3
[0091] Embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.
[0092] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:
[0093] S1, acquire video data;
[0094] S2, perform head target detection on the target object in the current frame image of the video data, and extract the head region image;
[0095] S3, input the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object.
[0096] S4, obtain multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model from the historical frame head region image of the target object;
[0097] S5. Based on the first model discrimination result and the second model discrimination result, determine the final wearing category of the mask worn by the target object.
[0098] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0099] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.
[0100] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.
[0101] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:
[0102] S1, acquire video data;
[0103] S2, perform head target detection on the target object in the current frame image of the video data, and extract the head region image;
[0104] S3, input the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object.
[0105] S4, obtain multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model from the historical frame head region image of the target object;
[0106] S5. Based on the first model discrimination result and the second model discrimination result, determine the final wearing category of the mask worn by the target object.
[0107] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.
[0108] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0109] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0114] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for identifying mask wearing, characterized in that, The method includes: Acquire video data; Perform head target detection on the target object in the current frame image of the video data, and extract the head region image; The head region image is input into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object. Obtain multiple second model discrimination results for the target object, wherein the second model discrimination results are mask wearing categories derived by the model from historical frame head region images of the target object; Based on the discrimination results of the first model and the discrimination results of the second model, the final wearing category of the mask for the target object is determined; The determination of the final mask-wearing category of the target object based on the first model discrimination result and the second model discrimination result includes: acquiring multiple second model discrimination results and the second mask-wearing category that appears most frequently in the first model discrimination result; determining whether the proportion of the second mask-wearing category is greater than a preset proportion; if the proportion of the second mask-wearing category is greater than the preset proportion, then using the second mask-wearing category as an intermediate logical result of the head region image of the current frame; and determining the final mask-wearing category of the target object based on the intermediate logical result. The process of determining the final mask-wearing category of the target object based on the intermediate logic result includes: determining whether the intermediate logic result indicates an uncertain wearing status; if the intermediate logic result indicates an uncertain wearing status, then using the final wearing category of the target object's previous frame head region image; if the intermediate logic result does not indicate an uncertain wearing status, then updating the final mask-wearing category of the target object to the intermediate logic result.
2. The method according to claim 1, characterized in that, The head region image is input into the mask-wearing recognition model to obtain the first model discrimination result of the head region image, including: The head region image is input into the mask wearing recognition model to obtain the confidence scores of multiple mask wearing categories corresponding to the head region image. Obtain the first mask-wearing category with the highest confidence level, and determine whether the highest confidence level is greater than a preset threshold; If the maximum confidence level is greater than the preset threshold, the first mask wearing category is taken as the first model's discrimination result; if the maximum confidence level is less than or equal to the preset threshold, the first model's discrimination result is determined to be that the wearing status is uncertain.
3. The method according to claim 1, characterized in that, After determining the final mask-wearing category of the target object based on the discrimination results of the first model and the second model, the method further includes: If the final wearing category is determined to be not wearing a mask, then the head area image is captured, and the monitoring address, occurrence time, duration of not wearing a mask, and the person ID of the target are recorded.
4. The method according to claim 1, characterized in that, Before inputting the head region image into the mask-wearing recognition model to obtain the first model discrimination result of the head region image, the method further includes: Obtain samples of mask-wearing; Sample labels are set for mask wearing samples, wherein the sample labels include those indicating that the mask has been worn correctly, has not been worn correctly, has not been worn, and the wearing status is uncertain; The initial model is trained using the mask-wearing samples and the following loss function to obtain the mask-wearing recognition model: Where m is the number of mask-wearing samples, n is the number of multi-label categories, and y i Let be the label value corresponding to the i-th label, where the label value is either 0 or 1. Let P be the probability value predicted by the mask-wearing recognition model for each label category, where the probability value ranges from [0, 1]. Let i be the sum of the cross-entropy of all labels from i=1 to i=n.
5. The method according to claim 1, characterized in that, Performing head target detection on the target object in the current frame image of the video data and extracting the head region image includes: Obtain the head region obtained after head target detection; Expand the area of the head region outward and extract the head region image from the current frame image.
6. A mask wearing recognition device, characterized in that, include: The acquisition module is used to acquire video data; The detection module is used to detect human head targets in the current frame image of the video data and extract the head region image; The first discrimination module is used to input the head region image into the mask wearing recognition model to obtain the first model discrimination result of the head region image. The first model discrimination result is the mask wearing category obtained by the model for the current frame head region image of the target object. The acquisition module is further configured to acquire multiple second model discrimination results of the target object, wherein the second model discrimination results are mask wearing categories derived by the model from historical frame head region images of the target object; The second discrimination module is used to determine the final wearing category of the mask worn by the target object based on the discrimination results of the first model and the discrimination results of the second model; The second discrimination module includes a second discrimination unit, which is used to acquire multiple second model discrimination results and the second mask wearing category that appears most frequently in the first model discrimination results; determine whether the proportion of the second mask wearing category is greater than a preset proportion; if the proportion of the second mask wearing category is greater than the preset proportion, then the second mask wearing category is used as the intermediate logical result of the head region image of the current frame; and determine the final mask wearing category of the target object based on the intermediate logical result. The second discrimination module further includes a third discrimination unit, used to determine whether the intermediate logic result is an uncertain wearing status; if the intermediate logic result is an uncertain wearing status, the final wearing category of the previous frame head region image of the target object is used; if the intermediate logic result is not an uncertain wearing status, the final wearing category of the target object's mask is updated to the intermediate logic result.
7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor for executing the method steps of any one of claims 1 to 5 by running a program stored in memory.
8. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program, when executed, performs the method steps of any one of claims 1 to 5.