Method and device for detecting human face and human face mask wearing based on single chip microcomputer system

By setting and moving the detection window in the microcontroller system, using the face detection accelerator and the mask detection accelerator for parallel cascade decision tree detection, the problem of face and face mask wearing detection without contact is solved, and efficient and accurate Automatic monitoring.

CN114463811BActive Publication Date: 2024-11-22SHENZHEN UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210054183.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-18
Publication Date
2024-11-22
Estimated Expiration
2042-01-18

AI Technical Summary

Technical Problem

The existing technology is difficult to achieve untouched face and face mask wear testing, and there is a lack of efficient monitoring solutions in scenarios such as restaurant kitchens, operating vehicle driver mask wear supervision, laboratories, and food processing plants.

Method used

Using the face and face mask wear detection method based on the microcontroller system, by obtaining video images, setting and moving the detection window, the face detection accelerator unit and the face mask detection accelerator unit perform parallel cascade detection of the decision tree cascade detection tree to achieve Quick test of the wear of human face or face masks.

Benefits of technology

It realizes untouched face and face mask wear detection, improves detection efficiency and accuracy, and is suitable for automatic monitoring needs in various scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114463811B_ABST
    Figure CN114463811B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for detecting human face and human face mask wearing based on a single-chip microcomputer system, the method comprising: obtaining a captured video image, extracting any image frame from the video image as a target image frame to be detected; setting two detection windows in the image area of ​​the target image frame and using them as a group of parallel detection windows, moving the parallel detection windows to traverse all image areas of the target image frame; using a human face detection accelerator unit and a human face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas respectively, to obtain a target object, the target object includes a human face or a human face mask wearing; and outputting the detection information of the human face and the human face mask wearing respectively. The technical scheme of the present invention can realize the unmanned contact detection of the human face and the human face mask wearing, improve the detection speed of the human face and the human face mask wearing, and can also improve the accuracy and reliability of the detection results.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of deep learning and edge computing technology, and in particular to a method, device and readable storage medium for detecting human face and human face mask wearing based on a single-chip microcomputer system. Background Art

[0002] With the rapid development of fifth-generation mobile network (5G) communication technology and artificial intelligence (AI), the Internet of Things (IoT) and edge computing have brought a new application revolution to our daily lives. Due to its low power consumption, small size, and high performance, single-board computers (SBCs) are widely used in IoT scenarios and have become the main carrier of edge accelerators such as artificial intelligence chips. The literature "T.-H.Jung, B.Cates, I.-K. Choi, S.-H.Lee, and J.-M.Choi, "Multi-Camera-Based Person Recognition System for Autonomous Tractors," Designs, vol.4, no.4, p.54, Dec.2020" proposes a method for implementing human detection on tractors based on SBC'NVIDIA Jetson AGX Xavier' and deep learning algorithm YOLOv3 to ensure the safe operation of tractors. It can be seen that when SBC is combined with various edge accelerators, we can run many high-performance AI algorithms on it, providing unlimited possibilities for realizing powerful intelligent applications.

[0003] How to combine the above AI algorithms with face and face mask wearing detection to achieve functions such as contactless monitoring, reducing crowds, and efficient reminders without the need for staff to be present on site has become a technical problem that needs to be solved in the industry. In addition, face and face mask wearing detection is also of great significance in scenarios such as restaurant kitchens, mask wearing supervision for operating vehicle drivers, laboratories, and food processing plants.

[0004] In view of this, it is necessary to further improve the current face and face mask wearing detection technology. Summary of the invention

[0005] In order to solve at least one of the above technical problems, the main purpose of the present invention is to provide a method, device and readable storage medium for detecting human face and human face mask wearing based on a single-chip microcomputer system.

[0006] To achieve the above purpose, the first technical solution adopted by the present invention is: to provide a method for detecting a human face and a human face mask wearing based on a single-chip microcomputer system, comprising:

[0007] Acquire the captured video image, and extract any image frame from the video image as the target image frame to be detected;

[0008] Two detection windows are set in the image area of ​​the target image frame as a group of parallel detection windows, and the parallel detection windows are moved to traverse all image areas of the target image frame;

[0009] Using a face detection accelerator unit and a face mask detection accelerator unit, respectively, parallel cascade decision tree cascade detection is performed on the image grayscale information of all parallel detection window areas to obtain a target object, wherein the target object includes a face or a face wearing a mask;

[0010] Output the detection information of the face and the face mask respectively.

[0011] The number of cascaded decision trees in the parallel cascade decision tree is 468 and the depth of each decision tree is 6. The parallel cascade decision tree cascade detection is performed according to the moving position of the parallel detection window in the image area of ​​the target image frame, specifically including:

[0012] Obtain the image grayscale information of any detection window area that requires decision making;

[0013] Determine whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree;

[0014] If yes, continue to detect all binary decision depths of the binary decision tree of the next cascade of the image grayscale information until all cascaded 468 binary decision trees are passed;

[0015] When the image grayscale information of the detection window area passes all levels of binary decision tree level detection, it is determined that the detection window area contains the target object.

[0016] The step of determining whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree specifically includes:

[0017] Extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree;

[0018] When the extracted confidence is greater than the confidence threshold corresponding to the first-level binary decision tree, it is determined that the image grayscale information of the detection window area passes the detection of the first-level binary decision tree.

[0019] After extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree, the method further includes:

[0020] When the extracted confidence is less than or equal to the confidence threshold of the corresponding first-level binary decision, it is determined that the image grayscale information of the detection window area has not passed the detection of the first-level binary decision tree, and the cascade detection is directly terminated and it is determined that the detection window area does not contain the target object.

[0021] The moving of the parallel detection window to traverse all image areas of the target image frame specifically includes:

[0022] Marking the initial coordinates, size and window step of the parallel detection window, wherein the window step includes a horizontal window step and a vertical window step;

[0023] The parallel detection window is laterally moved according to the initial coordinates of the parallel detection window and the horizontal window moving step length, and it is determined whether the horizontal window moving of the parallel detection window is completed;

[0024] When the parallel detection window is moved horizontally, the parallel detection window is moved vertically according to the initial coordinates of the parallel detection window and the vertical window moving step, and the horizontal window is continued to be moved horizontally according to the window moving coordinates and the horizontal window moving step of the parallel detection window that has been moved vertically, so as to traverse all image areas of the target image frame;

[0025] The detection information of the face and the face mask respectively outputted specifically includes:

[0026] The coordinate information of the detection results of the face and the face mask worn and the parallel detection window size information are output respectively.

[0027] Wherein, after traversing all image areas of the target image frame, the method further includes:

[0028] Proportionally enlarge the initial coordinates, size and window step length of the parallel detection window;

[0029] Determine whether the size of the parallel detection window after enlargement is greater than a preset value;

[0030] If the size of the parallel detection window is smaller than the preset value, the parallel detection window is horizontally and vertically shifted according to the enlarged window shifting step size and the initial coordinates to traverse all image areas of the target image frame;

[0031] Performing the next equal-proportional enlargement of the parallel detection window until the size of the equal-proportional enlarged parallel detection window is greater than or equal to the preset value, and then stopping the window shifting process of the equal-proportional enlarged parallel detection window; and

[0032] Filters the overlapping detection objects in the parallel detection window.

[0033] After obtaining the target object, the method further includes:

[0034] Counting the number of the target objects;

[0035] The detection information of the face and the face mask respectively outputted specifically includes:

[0036] The coordinate information of the detection results of the face and the face mask worn and the detection window size information of the target object's location are output respectively.

[0037] To achieve the above purpose, the second technical solution adopted by the present invention is: to provide a human face and human face mask wearing detection device based on a single chip microcomputer system, comprising:

[0038] An acquisition module is used to acquire the captured video image and extract any image frame from the video image as a target image frame to be detected;

[0039] A traversal module, used for setting two detection windows in the image area of ​​the target image frame as a group of parallel detection windows, and moving the parallel detection windows to traverse all image areas of the target image frame;

[0040] A detection module, used to use a face detection accelerator unit and a face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas respectively, to obtain a target object of interest, wherein the target object includes a face or a face wearing a mask;

[0041] The output module is used to output the detection information of the face and the face mask respectively.

[0042] To achieve the above-mentioned purpose, the third technical solution adopted by the present invention is: to provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps in the above-mentioned method are implemented.

[0043] To achieve the above objectives, the fourth technical solution adopted by the present invention is: providing a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0044] The technical solution of the present invention adopts the method of first acquiring a captured video image, extracting any image frame from the video image as a target image frame to be detected, then setting two detection windows in the image area of ​​the target image frame as a group of parallel detection windows, moving the parallel detection windows to traverse all image areas of the target image frame, and then using a face detection accelerator unit and a face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas to obtain a target object, wherein the target object includes a face or a face wearing a mask, and finally outputting the detection information of the face and the face wearing a mask respectively, so as to realize unmanned contact detection of the face and the face wearing a mask, improve the detection speed of the face and the face wearing a mask, and also improve the accuracy and reliability of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0046] Figure 1 This is a method flow chart of a method for detecting a human face and a human face mask wearing method based on a single chip microcomputer system according to an embodiment of the present invention;

[0047] Figure 2 This is a specific flow chart of a method for detecting a human face and a human face mask wearing based on a single-chip microcomputer system according to an embodiment of the present invention;

[0048] Figure 3 Training ROC curve for the face detection decision tree cascade parameter model of the present invention;

[0049] Figure 4 Training ROC curve for the decision tree cascade parameter model of face mask wearing detection of the present invention;

[0050] Figure 5 A detection flow chart of parallel cascade decision tree cascade detection according to an embodiment of the present invention;

[0051] Figure 6 This is a rendering of the face detection and face mask wearing detection of the present invention;

[0052] Figure 7 It is a module block diagram of a human face and human face mask wearing detection device based on a single chip microcomputer system of the present invention;

[0053] Figure 8 The figure is a module block diagram of an electronic device according to an embodiment of the present invention.

[0054] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be noted that the descriptions of "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0057] Different from the prior art that most places require staff to detect human faces and face mask wearing, and cannot realize unattended face and face mask wearing detection, the present invention provides a face and face mask wearing detection method based on a single-chip microcomputer system, which aims to realize unmanned face and face mask wearing detection, which can improve the detection efficiency, and at the same time can also improve the accuracy and reliability of the detection results.

[0058] Please refer to Figure 1 and Figure 2 , Figure 1 This is a method flow chart of a method for detecting a human face and a human face mask wearing based on a single chip microcomputer system according to a first embodiment of the present invention. Figure 2 This is a specific flow chart of the face and face mask wearing detection method based on the single-chip microcomputer system of the first embodiment of the present invention. In the embodiment of the present invention, the face and face mask wearing detection method based on the single-chip microcomputer system, the single-chip microcomputer system includes an arm Cortex-M3 processor, a face detection accelerator unit, and a face mask detection accelerator unit. When the face detection accelerator unit and the face mask detection accelerator unit are triggered, the arm Cortex-M3 processor enters a dormant state. When the face detection accelerator unit and the face mask detection accelerator unit complete the detection, the arm Cortex-M3 processor resumes work. In the specific implementation, two interrupt signals accStart() and WaitForReturn() can be configured to achieve it. The method specifically includes the following steps:

[0059] S110, obtain the captured video image, and extract any image frame from the video image as the target image frame to be detected. Specifically, in this scheme, a camera is used to capture video images in real time, and then the captured video images are stored, and any image frame is extracted from the video image as the target image frame to be detected by the face detection accelerator unit and the face mask detection accelerator unit. Before step S110, it also includes initializing the detection result storage array rcsq[], which is a group of four bits, and stores the horizontal coordinate, vertical coordinate, area side length and confidence from low to high, as well as setting the maximum side length of each window shifting area and the window shifting step length of each window shifting. In this scheme, the detection window is a square, and the side length of the area is the size of the parallel detection window.

[0060] S120, setting two detection windows in the image area of ​​the target image frame as a group of parallel detection windows, and moving the parallel detection windows to traverse all image areas of the target image frame. Specifically, after selecting any target image frame, setting two detection windows of the same size and arranged side by side in the image area of ​​the target image frame as a group of parallel detection windows. That is, in this solution, the target image frame can be divided into a number of detection units by using parallel detection windows and moving the parallel detection windows, and then all detection units are detected to detect the target image frame.

[0061] Specifically, the moving the parallel detection window to traverse all image areas of the target image frame specifically includes:

[0062] Marking the initial coordinates, size and window step of the parallel detection window, wherein the window step includes a horizontal window step and a vertical window step;

[0063] The parallel detection window is laterally moved according to the initial coordinates of the parallel detection window and the horizontal window moving step length, and it is determined whether the horizontal window moving of the parallel detection window is completed;

[0064] When the lateral window shifting of the parallel detection window is not completed, the lateral window shifting of the parallel detection window continues;

[0065] When the parallel detection window is moved horizontally, the parallel detection window is moved vertically according to the initial coordinates of the parallel detection window and the vertical window moving step, and the horizontal window is continued to be moved horizontally according to the window moving coordinates and the horizontal window moving step of the parallel detection window that has been moved vertically, so as to traverse all image areas of the target image frame;

[0066] The separately outputting detection information of the human face and the human face wearing a mask specifically includes: separately outputting coordinate information of the detection results of the human face and the human face wearing a mask and parallel detection window size information.

[0067] Specifically, the above-mentioned parallel detection window includes two detection windows, which respectively record the initial coordinates and window moving steps of the two detection windows, and the initial coordinates include horizontal coordinates and vertical coordinates. This scheme realizes the detection of all image areas of the target image frame by moving the parallel detection window horizontally and vertically. When the window is specifically moved, the parallel detection window is located in the upper left corner area of ​​the target image frame. The parallel detection window is first moved horizontally, and then the parallel detection window is moved vertically after the horizontal window is moved horizontally. After the vertical window is moved, the parallel detection window continues to move horizontally for the next round until all image areas of the target image frame are traversed, and the horizontal and vertical window moves are completed. After the parallel detection window detection is completed, the coordinate information of the detection results of the face and the face mask and the parallel detection window size information can be output. The coordinate information corresponds to the position of the image area in the target image frame, and the size information corresponds to the side length of the image area, so as to facilitate the rapid search for the target position.

[0068] Furthermore, after traversing all image areas of the target image frame, the method further includes:

[0069] Proportionally enlarge the initial coordinates, size and window step length of the parallel detection window;

[0070] Determine whether the size of the parallel detection window after enlargement is greater than a preset value;

[0071] If the size of the parallel detection window is smaller than the preset value, the parallel detection window is horizontally and vertically shifted according to the enlarged window shifting step size and the initial coordinates to traverse all image areas of the target image frame;

[0072] Performing the next equal-proportional enlargement of the parallel detection window until the size of the equal-proportional enlarged parallel detection window is greater than or equal to the preset value, and then stopping the window shifting process of the equal-proportional enlarged parallel detection window; and

[0073] Filters the overlapping detection objects in the parallel detection window.

[0074] In this embodiment, in addition to horizontally and vertically shifting the parallel detection window of the set side length, it also includes proportionally enlarging the parallel detection window, that is, proportionally enlarging the size of the parallel detection window of the preset side length, and then horizontally and vertically shifting the parallel detection window after proportional enlargement. For the specific window shifting method, please refer to the above embodiment, which will not be repeated here. In order to better enlarge the parallel detection window, this scheme also includes comparing the size of the parallel detection window after proportional enlargement with the preset value. If the size of the parallel detection window after proportional enlargement is less than the preset value, the initial coordinates and the window shifting step of the parallel detection window are continued to be horizontally and vertically moved. Move the parallel detection window to traverse all image areas of the target image frame, and the horizontal and vertical window shifting ends, indicating that the face and face mask wearing detection of the proportionally enlarged parallel detection window is completed. At this time, this scheme starts the next round of proportionally enlarging the size of the parallel detection window, and then continues to judge the size of the enlarged parallel detection window and the preset value. If the size of the parallel detection window after the proportional enlargement of this round is less than or equal to the preset value, stop the window shifting process of the parallel detection window enlarged this time. The purpose of vertical window shifting is to ensure that the detection target image frame area completely includes the face and the face mask wearing area as much as possible, and remove other areas to improve the detection and recognition efficiency and detection accuracy. Considering that the detected target objects may overlap after the size of the detection window is enlarged proportionally many times, this solution also needs to filter the overlapping detection target objects to output a unique detection result for a single target object to improve the accuracy of the detection result. When filtering overlapping detection targets, additional overlapping targets detected in each round can also be filtered.

[0075] Specifically, after obtaining the target object, the method further includes:

[0076] Counting the number of the target objects;

[0077] The detection information of the face and the face mask respectively outputted specifically includes:

[0078] The coordinate information of the detection results of the face and the face mask are output respectively, as well as the detection window size information of the target object. This solution can quickly count the number of target objects and quickly locate the target objects.

[0079] S130, using the face detection accelerator unit and the face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas respectively, to obtain a target object, wherein the target object includes a face or a face mask. Specifically, the face detection accelerator unit and the face mask detection accelerator unit can perform parallel cascade decision tree cascade detection on the parallel detection window area respectively, and output the target object. When the face detection accelerator unit and the face mask detection accelerator unit do not detect the target object, a prompt message that there is no target object is output.

[0080] Before performing parallel cascade decision tree cascade detection on the image grayscale information of the above parallel detection window area, this scheme also needs to train the face detection decision tree cascade parameter model. The Caltech_WebFaces dataset is selected to estimate the position and proportion of each face based on simple anthropometric relationships and the position coordinate information of both eyes, and derive the bounding box of each face as a positive sample. Based on this, all approximately 20,000 framed face images are extracted from the dataset, and 15 positive training samples are generated from each front face through small random perturbations of scale and position. In the end, a total of approximately 300,000 positive samples are generated. For each stage of training and learning, this design extracts 300,000 negative samples from a large number of images that do not contain any faces by collecting areas that have not been discarded in any stage of previous learning. The final face detection parameter model size is only 234KB. Similarly, for the training of the face mask detection decision tree cascade parameter model, this design selected the Face_Mask_Detection dataset in Kaggle. Different from the registration method of face detection, this design found through a large number of tests that when the extracted positive sample area contains the features of the face mask and human eyes, additional face information is needed to ensure the accuracy of face mask detection (if not included, the mask will be used as the detection subject, which deviates from the original design intention of the product). Therefore, it is assumed that the eyes of the face are at 1 / 10 of the frame column direction, with the midpoint of the eye as the center of the circle, and the distance from this point to the chin as the radius to extract positive samples from the face mask area. This will include more human features such as ears and eyebrows, thereby greatly reducing the error rate of the trained model. Similarly, in the subsequent training process, a total of about 50,000 positive samples and 50,000 negative samples were extracted for training. The final face mask detection parameter model size is only 53KB. Finally, this design uses a script file to convert the two parameter models into a .coe file for ROM initialization of Xilinx FPGA. The two together occupy no more than 1M of storage space, which allows the parameter files to be directly placed in the FPGA BlockRAM memory and called by each group of parallel cascade decision tree cascade acceleration units. Please refer to Figures 3 and Figure 4 , Figure 3To train the ROC curve for the face detection decision tree cascade parameter model, Figure 4 The ROC curve is trained for the decision tree cascade parameter model for face mask wearing detection. Figure 3 and Figure 4 It can be seen that the ROC curves of the decision tree cascade parameter model training for face detection and face mask detection both have good convex characteristics, that is, they both have large AUC values, which indicates that the two decision tree cascade classifiers trained have extremely high classification performance.

[0081] In the cascade detection using parallel cascade decision trees, this scheme adopts multi-threaded parallel window shifting, and uses two decision tree cascade algorithm units in parallel to form a parallel cascade decision tree cascade algorithm acceleration unit to process a frame of image. Each group of parallel acceleration units jointly completes the face detection or face mask detection of an image, and stores the detection results to the corresponding address for the Cortex-M3 processor to complete the subsequent processing of the detection results. The results after the above detection can be received through the set pointers t1 and t2. The face detection accelerator unit and the face mask detection accelerator unit can independently retrieve the image data in DDR3. In addition, this design is equipped with two independent Block RAMs for the two accelerator units to store their respective decision tree parameter files, so that the two do not affect each other when performing detection work on the same frame, are independent of each other and can be processed in parallel.

[0082] S140: Output the detection information of the human face and the human face mask respectively.

[0083] Specifically, the above detection information may be detailed information output via a serial port, processed image information output via an HDMI interface, or detection result information output via an LED.

[0084] Please refer to Figure 5 , Figure 5 The parallel cascade decision tree cascade detection detection flow chart of the present invention. In a specific embodiment, the number of cascaded decision trees in the parallel cascade decision tree is 468 and the depth of each decision tree is 6. The parallel cascade decision tree cascade detection is performed according to the moving position of the parallel detection window in the image area of ​​the target image frame, specifically including:

[0085] Obtain the image grayscale information of any detection window area that requires decision making;

[0086] Determine whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree;

[0087] If yes, continue to detect all binary decision depths of the binary decision tree of the next cascade of the image grayscale information until all cascaded 468 binary decision trees are passed;

[0088] When the image grayscale information of the detection window area passes the detection through all levels of the binary decision tree, it is determined that the detection window area contains the target object.

[0089] Specifically, this scheme adopts a deep learning target detection algorithm based on pixel intensity comparison (Pixel Intensity Comparison-based Object detection, PICO), uses point pair features for target detection, compares the pixel intensities of two pixels, and requires shorter detection time, small model size, high detection rate, and no need for image preprocessing. The PICO algorithm mainly trains the model by extracting features from the selected area of ​​the image, so this design adopts a series of face registration (key point positioning) strategies when selecting and preprocessing the data set. For face detection, this design ensures that the selected area of ​​each training image contains facial features as much as possible; and for face mask detection, this design ensures that the selected area contains a complete mask and contains as many facial features as possible, while also minimizing the redundant information in the selected area except for the face and mask.

[0090] When the above-mentioned face detection decision tree cascade parameter model and face mask wearing detection decision tree cascade parameter model are used to detect the parallel detection window area, the binary decision tree level and binary decision depth of the parallel cascade decision are first initialized. For this solution, there are 468 binary decision trees, and the depth of each decision tree is 6. During the specific binary decision, all binary decision depths are compared for the image grayscale information in the parallel detection window under the first-level binary decision tree. If all binary decision depth comparisons are passed, it means that the image grayscale information passes all binary decision depths of the first-level binary decision tree for detection. At this time, the image grayscale information continues to be detected at all binary decision depths of the next-level cascade binary decision tree until it passes the detection of all cascaded 468 binary decision trees. After the image grayscale information passes the detection of all binary decision tree levels, it is determined that the detection window area contains the target object, otherwise it is determined that the detection window does not contain the target object.

[0091] In order to indicate that the image grayscale information passes all binary decision depth detections, the present solution also introduces the confidence of the binary decision depth. The process of determining whether the image grayscale information in the detection window area passes all binary decision depth detections of the first-level binary decision tree specifically includes:

[0092] Extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree;

[0093] When the extracted confidence is greater than the confidence threshold corresponding to the first-level binary decision tree, it is determined that the image grayscale information of the detection window area passes the detection of the first-level binary decision tree.

[0094] In this embodiment, the depth of each decision tree of the above binary decision tree is 6. When performing binary decision depth comparison, the comparison results of each binary decision depth are accumulated, and the corresponding confidence is accumulated at the same time until the binary detection of all binary decision depths is completed. The confidence threshold of the first-level binary decision tree is extracted, and the accumulated confidence is compared with the confidence threshold. If the accumulated confidence is greater than the confidence threshold, it means that the first-level binary decision tree passes the test, and the test result is recorded as 1. At this time, the binary decision tree of the next level can be tested. It can be understood that each comparison of the binary decision depth has a corresponding confidence, and the test results of all depths of the first-level binary decision tree are the accumulation of the confidence corresponding to the test results of each depth.

[0095] In a specific embodiment, after extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree, the method further includes:

[0096] When the extracted confidence is less than or equal to the confidence threshold of the corresponding first-level binary decision, it is determined that the image grayscale information of the detection window area has not passed the detection of the first-level binary decision tree, and the cascade detection is directly terminated and it is determined that the detection window area does not contain the target object. The detection result is recorded as 0. At this time, the detection window can be excluded and subsequent detection can be exited.

[0097] In addition, the arm Cortex-M3 processor, as the CPU, coordinates and controls the operation of the entire SoC through C language programs, improving the efficiency of the collaborative work between various modules and ensuring the stability of the system. At the same time, when the SoC starts the two acceleration units to monitor the data collected by the camera, the Cortex-M3 processor will enter sleep mode to reduce system power consumption; and when the accelerator unit completes the detection, the Cortex-M3 processor will be awakened to quickly process the detection results, and the detection results can be presented in a variety of forms, which reduces the difficulty of SoC hardware development to a certain extent.

[0098] Please refer to Figure 6 , Figure 6This is a rendering of face detection and face mask wearing detection. In a specific test case, the MCU system performs frame processing on the image captured by the camera at the software level based on the detection results of the face detection acceleration unit and the face mask detection accelerator unit, and outputs the face detection status through the HDMI connection display. In actual applications, the output information can be used with a red frame to identify the face mask wearing detection status in the captured image, and a green frame to identify the face detection status in the captured image.

[0099] Table 1

[0100] Face Detection SoC The SoC implemented in this design Power consumption 1.46W 1.45W

[0101] Table 1 is the operating power consumption table of SoC. As can be seen from Table 1, the power consumption of the SoC implemented by this design is only 1.45W during stable operation. Its low power consumption is perfectly suitable for edge scenarios with highly energy-constrained energy consumption. While saving energy, the system can also run in a stable state for a longer time and complete more detection work, which has extremely high economic value.

[0102] Table 2

[0103] SoC working status SoC average detection frame rate Empty inspection 52FPS No frame processing 32FPS With picture frame 31FPS

[0104] Table 2 is a comparison table of average frame rate detection of SoC. Table 2 shows the comparison of average frame rate detection of the face and face mask wearing detection method based on deep learning algorithm and SoC system implemented in this design under different working conditions. It can be seen from Table 2 that the SoC implemented in this design can realize the face and face mask wearing detection function with high real-time performance, with an average of more than 30FPS. Although the frame processing of the image by the arm Cortex-M3 processor will slightly reduce the detection frame rate of the SoC, this disadvantage can be avoided in scenarios where video output is not required but only UART communication is required.

[0105] Compared with the prior art, the present invention proposes a method for realizing face mask wearing detection based on SoC of arm Cortex-M3 processor and deep learning accelerator. The method adopts Cortex-M3 processor as the central processing unit of the system, equipped with a self-designed deep learning accelerator, the accelerator adopts an improved PICO algorithm, and through parallel window processing and independently running hardware acceleration unit module, it can complete the rapid detection of whether the face is wearing a mask. The operation results show that the designed method can perform face mask wearing detection at an average speed of up to 37FPS on Xilinx Artix-7 FPGA, realizing high-performance, high-stability and low-power detection functions.

[0106] Please refer to Figure 7 , Figure 7 This is a module block diagram of a human face and human face mask wearing detection device based on a single chip microcomputer system according to a third embodiment of the present invention. In an embodiment of the present invention, the human face and human face mask wearing detection device based on a single chip microcomputer system comprises:

[0107] The acquisition module 110 is used to acquire the captured video image and extract any image frame from the video image as a target image frame to be detected;

[0108] A traversal module 120, configured to set two detection windows in the image region of the target image frame as a group of parallel detection windows, and move the parallel detection windows to traverse all image regions of the target image frame;

[0109] The detection module 130 is used to use the face detection accelerator unit and the face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas respectively to obtain the target object of interest, wherein the target object includes a face or a face wearing a mask;

[0110] The output module 140 is used to output the detection information of the face and the face mask wearing respectively.

[0111] Specifically, the number of cascaded decision trees in the parallel cascaded decision tree is 468 and the depth of each decision tree is 6. The detection module 130 is used for:

[0112] Obtain the image grayscale information of any detection window area that requires decision making;

[0113] Determine whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree;

[0114] If yes, continue to detect all binary decision depths of the binary decision tree of the next cascade of the image grayscale information until all cascaded 468 binary decision trees are passed;

[0115] When the image grayscale information of the detection window area passes all levels of binary decision tree level detection, it is determined that the detection window area contains the target object.

[0116] Specifically, the detection module 130 is further used for:

[0117] Extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree;

[0118] When the extracted confidence is greater than the confidence threshold corresponding to the first-level binary decision tree, it is determined that the image grayscale information of the detection window area passes the detection of the first-level binary decision tree.

[0119] Specifically, the detection module 130 is further used for:

[0120] When the extracted confidence is less than or equal to the confidence threshold of the corresponding first-level binary decision, it is determined that the image grayscale information of the detection window area has not passed the detection of the first-level binary decision tree, and the cascade detection is directly terminated and it is determined that the detection window area does not contain the target object.

[0121] Specifically, the traversal module 120 is also used for:

[0122] Marking the initial coordinates, size and window step of the parallel detection window, wherein the window step includes a horizontal window step and a vertical window step;

[0123] The parallel detection window is laterally moved according to the initial coordinates of the parallel detection window and the horizontal window moving step length, and it is determined whether the horizontal window moving of the parallel detection window is completed;

[0124] When the parallel detection window is moved horizontally, the parallel detection window is moved vertically according to the initial coordinates of the parallel detection window and the vertical window moving step, and the horizontal window is continued to be moved horizontally according to the window moving coordinates and the horizontal window moving step of the parallel detection window that has been moved vertically, so as to traverse all image areas of the target image frame; the output module 140 is specifically used for:

[0125] The coordinate information of the detection results of the face and the face mask worn and the parallel detection window size information are output respectively.

[0126] Specifically, the traversal module 120 is also used for:

[0127] Proportionally enlarge the initial coordinates, size and window step length of the parallel detection window;

[0128] Determine whether the size of the parallel detection window after enlargement is greater than a preset value;

[0129] If the size of the parallel detection window is smaller than the preset value, the parallel detection window is horizontally and vertically shifted according to the enlarged window shifting step size and the initial coordinates to traverse all image areas of the target image frame;

[0130] Performing the next equal-proportional enlargement of the parallel detection window until the size of the equal-proportional enlarged parallel detection window is greater than or equal to the preset value, and then stopping the window shifting process of the equal-proportional enlarged parallel detection window; and

[0131] Filters the overlapping detection objects in the parallel detection window.

[0132] Specifically, the traversal module 120 is also used for:

[0133] Counting the number of the target objects;

[0134] The output module 140 is specifically used for:

[0135] The coordinate information of the detection results of the face and the face mask worn and the detection window size information of the target object's location are output respectively.

[0136] See also Figure 8 , Figure 8 FIG. 1 is a block diagram of a module of an electronic device according to an embodiment of the present invention. The electronic device can be used to implement the method for detecting a human face and a human face mask based on a single chip microcomputer system in the aforementioned embodiment. Figure 8 As shown, the electronic device mainly includes: a memory 301, a processor 302, a bus 303, and a computer program stored in the memory 301 and executable on the processor 302. The memory 301 and the processor 302 are connected via the bus 303. When the processor 302 executes the computer program, the face and face mask wearing detection method based on the single-chip system in the aforementioned embodiment is implemented. The number of processors can be one or more.

[0137] The memory 301 may be a high-speed random access memory (RAM) memory, or a non-volatile memory, such as a disk memory. The memory 301 is used to store executable program codes, and the processor 302 is coupled to the memory 301 .

[0138] Furthermore, an embodiment of the present invention further provides a readable storage medium, which may be disposed in the electronic device in the above embodiments. The readable storage medium may be the above Figure 8 Memory in the illustrated embodiment.

[0139] The readable storage medium stores a computer program, and when the program is executed by the processor, the method for detecting the face and the face mask wearing based on the single-chip microcomputer system in the aforementioned embodiment is implemented. Furthermore, the computer storable medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0140] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0141] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.

[0143] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk and other media that can store program codes.

[0144] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the technical solution of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for detecting human face and human face mask wearing based on a single chip microcomputer system, characterized in that: The method for detecting the wearing of a human face mask based on a single chip microcomputer system comprises: Acquire the captured video image, and extract any image frame from the video image as the target image frame to be detected; Two detection windows are set in the image area of ​​the target image frame as a group of parallel detection windows, and the parallel detection windows are moved to traverse all image areas of the target image frame; A face detection accelerator unit and a face mask detection accelerator unit are used to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas to obtain a target object. In the cascade detection using the parallel cascade decision tree, multi-threaded parallel window shifting is adopted, and two decision tree cascade algorithm units are used in parallel to form a parallel cascade decision tree cascade algorithm acceleration unit to process a frame of image. Each group of parallel acceleration units jointly completes face detection or face mask detection of an image; the target object includes a face or a face mask; Output the detection information of human face and human face mask respectively; The number of cascaded decision trees in the parallel cascade decision tree is 468 and the depth of each decision tree is 6. The parallel cascade decision tree cascade detection is performed according to the moving position of the parallel detection window in the image area of ​​the target image frame, specifically including: Obtain the image grayscale information of any detection window area that requires decision making; Determine whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree; If yes, continue to detect all binary decision depths of the binary decision tree of the next cascade of the image grayscale information until all cascaded 468 binary decision trees are passed; When the image grayscale information of the detection window area passes the binary decision tree level detection of all levels, determining that the detection window area contains the target object; The step of determining whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree specifically includes: Extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree; When the extracted confidence is greater than the confidence threshold corresponding to the first-level binary decision tree, it is determined that the image grayscale information of the detection window area passes the detection of the first-level binary decision tree.

2. The method for detecting human face and human face mask wearing based on a single chip microcomputer system as claimed in claim 1, characterized in that: After extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree, the method further includes: When the extracted confidence is less than or equal to the confidence threshold of the corresponding first-level binary decision, it is determined that the image grayscale information of the detection window area has not passed the detection of the first-level binary decision tree, and the cascade detection is directly terminated and it is determined that the detection window area does not contain the target object.

3. The method for detecting human face and human face mask wearing based on a single chip microcomputer system as claimed in claim 1, characterized in that: The moving the parallel detection window to traverse all image areas of the target image frame specifically includes: Marking the initial coordinates, size and window step of the parallel detection window, wherein the window step includes a horizontal window step and a vertical window step; The parallel detection window is laterally moved according to the initial coordinates of the parallel detection window and the horizontal window moving step length, and it is determined whether the horizontal window moving of the parallel detection window is completed; When the parallel detection window is moved horizontally, the parallel detection window is moved vertically according to the initial coordinates of the parallel detection window and the vertical window moving step, and the horizontal window is continued to be moved horizontally according to the window moving coordinates and the horizontal window moving step of the parallel detection window that has been moved vertically, so as to traverse all image areas of the target image frame; The detection information of the face and the face mask respectively outputted specifically includes: The coordinate information of the detection results of the face and the face mask worn and the parallel detection window size information are output respectively.

4. The method for detecting human face and human face mask wearing based on a single chip microcomputer system as claimed in claim 2, characterized in that: After traversing all image areas of the target image frame, the method further includes: Proportionally enlarge the initial coordinates, size and window step length of the parallel detection window; Determine whether the size of the parallel detection window after enlargement is greater than a preset value; If the size of the parallel detection window is smaller than the preset value, the parallel detection window is horizontally and vertically shifted according to the enlarged window shifting step size and the initial coordinates to traverse all image areas of the target image frame; Performing the next equal-proportional enlargement of the parallel detection window until the size of the equal-proportional enlarged parallel detection window is greater than or equal to the preset value, and then stopping the window shifting process of the equal-proportional enlarged parallel detection window; and Filters the overlapping detection objects in the parallel detection window.

5. The method for detecting human face and human face mask wearing based on a single chip microcomputer system as claimed in claim 4, characterized in that: After obtaining the target object, the method further includes: Counting the number of the target objects; The detection information of the face and the face mask respectively outputted specifically includes: The coordinate information of the detection results of the face and the face mask worn and the detection window size information of the target object's location are output respectively.

6. A human face and human face mask wearing detection device based on a single chip microcomputer system, characterized in that: The human face and human face mask wearing detection device based on the single chip microcomputer system comprises: An acquisition module is used to acquire the captured video image and extract any image frame from the video image as a target image frame to be detected; A traversal module, used for setting two detection windows in the image area of ​​the target image frame as a group of parallel detection windows, and moving the parallel detection windows to traverse all image areas of the target image frame; A detection module is used to use a face detection accelerator unit and a face mask detection accelerator unit to perform parallel cascade decision tree cascade detection on the image grayscale information of all parallel detection window areas respectively to obtain a target object of interest. In the cascade detection using the parallel cascade decision tree, multi-threaded parallel window shifting is adopted, and two decision tree cascade algorithm units are used in parallel to form a parallel cascade decision tree cascade algorithm acceleration unit to process a frame of image, and each group of parallel acceleration units jointly completes face detection or face mask detection of an image; the target object includes a face or a face mask; wherein the number of cascaded decision trees in the parallel cascade decision tree is 468 and the depth of each decision tree is 6, and the parallel cascade decision tree cascade detection is performed according to the moving position of the parallel detection window in the image area of ​​the target image frame, specifically including: Obtain the image grayscale information of any detection window area that requires decision making; Determine whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree; If yes, continue to detect all binary decision depths of the binary decision tree of the next cascade of the image grayscale information until all cascaded 468 binary decision trees are passed; When the image grayscale information of the detection window area passes the binary decision tree level detection of all levels, determining that the detection window area contains the target object; The step of determining whether the image grayscale information of the detection window area passes all binary decision depth detections of the first-level binary decision tree specifically includes: Extracting the confidence corresponding to the binary decision depth accumulation result of the first-level binary decision tree and the confidence threshold corresponding to the first-level binary decision tree; When the extracted confidence is greater than the confidence threshold of the corresponding first-level binary decision tree, determining that the image grayscale information of the detection window area passes the detection of the first-level binary decision tree; The output module is used to output the detection information of the face and the face mask respectively.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of claims 1 to 5 when executing the computer program.

8. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Cascaded convolutional neural network based human face occlusion detection method

    CN105868689A

  • Personnel mask wearing state detection method

    CN111582068A

  • Method for detecting object

    KR1020150109901A

  • Method for accelerating operations and accelerator apparatus

    US20210241082A1