Image quality distortion detection, identification system and method and automatic tracking pusher

Abstract

The present application belongs to the field of mask wearing detection technology, and provides an image quality distortion detection, identification system and method, an automatic tracking pusher, an automatic tracking device; a remote controller used in conjunction with the automatic tracking device, the remote controller including a transmitter and a receiver, wherein the receiver is connected to the main control system of the automatic tracking device, and the main control system of the automatic tracking device is connected to an openmv camera; a body, the body adopts a detachable connection, and the body is driven by a drive system; an openmv camera provided on the body; the remote controller also includes a single-chip microcomputer. The present invention has two modes: automatic tracking and manual adjustment tracking, which greatly improves its applicability and makes its application range wider. By using the automatic tracking device and the openmv camera in conjunction with each other, it is possible to lock the person who is not wearing a mask. The present invention can automatically track and warn the person who is not wearing a mask while recognizing that the person is not wearing a mask; it has wider applicability.

Description

Technical Field

[0001] The present invention belongs to the technical field of mask wearing detection, and in particular relates to an image quality distortion detection and recognition system and method and an automatic tracking pusher. Background Art

[0002] With the rapid development of computer technology in recent years, video image target detection technology based on deep learning has gradually developed.

[0003] Currently, most mask wear recognition systems are based on algorithms, using the Yolo model in object detection and cameras. These systems rely solely on the camera to identify a fixed image and use the model to make probabilistic judgments about the people in the image. However, this approach results in a small monitoring area, making it impossible to identify individuals and providing assistance and support to those not wearing masks.

[0004] In actual application, when detecting whether a person is wearing a mask, the image obtained by the camera is distorted and blurred because the person being detected is in a state of rapid movement, resulting in errors in detection and inability to make accurate judgments.

[0005] In view of this, how to solve the defects in the above technical solutions has become one of the urgent problems to be solved in the field of mask wearing detection technology. Summary of the Invention

[0006] In view of the problems existing in the background technology, the present invention provides a mask wearing detection and recognition system, which includes a main system, a main control system of an automatic tracking device; an image acquisition unit; an image processing unit; an image recognition unit and a feature detection unit, and

[0007] The detection method of the image quality distortion detection and identification system is characterized by comprising the following steps:

[0008] Step S1: first receive a pedestrian mask wearing detection request sent by the main system;

[0009] Step S2: determine the candidate area (using sliding windows of different sizes to frame a certain image area in the image as the candidate area).

[0010] Use the OpenMV camera to obtain public places and personnel flow monitoring videos to obtain original sample images;

[0011] Step S3, extracting visual features related to the candidate regions of the original sampled image;

[0012] Step S4, processing the acquired original sampled image;

[0013] S4.1, use the built-in img.to_grayscale function of the openmv camera to convert the image into a grayscale image, and use the image.statistics.stdev function to obtain the standard deviation of the image grayscale value. Use the grayscale value variance formula to perform a square operation to obtain the grayscale value variance;

[0014] S4.2, if the grayscale value variance of the image is higher than the set threshold of 300, the frame image is considered to have high definition and is used, and the process proceeds to the next step S5;

[0015] S4.3, if the grayscale value variance of the image is lower than the set threshold, the image frame is considered blurred and discarded, and the process goes to step S2 to recapture the image.

[0016] Among them, the image gray value variance formula is: Where g is the average grayscale value of the image pixels; s is the grayscale value variance of the image pixel grayscale value; is the number of horizontal pixels in the image; The number of vertical pixels in the image; Pixel The gray value of are the horizontal and vertical coordinates of the pixel points respectively;

[0017] Step S5: Use a classifier to perform identification and analyze and pre-process the extracted data; for example, the location coordinates of the pedestrian and whether the pedestrian is wearing a mask;

[0018] S5.1, using the threshold editor of the machine vision system built into the OpenMV camera, obtain the color threshold to be detected. Using the binary image filtering method, for the RGB565 image, adjust the maximum and minimum values of the three parameters L, A, and B in the LAB color space to obtain a specific color threshold list thresholds;

[0019] S5.2, identify the mask markers in each frame of the image obtained by the openmv camera, and divide the outer image into foreground and the other images into background; according to the threshold calculation formula,

[0020] If the pixel value falls within the threshold range T, the pixel is classified as foreground;

[0021] If the pixel value does not fall within the threshold range T, the pixel is classified as background;

[0022] The formula is as follows: in, Indicates that the image is The L, A, and B thresholds at , T represents the range of values that the three parameters L, A, and B can take;

[0023] S5.3: After performing the binary filtering, the find_blobs function built into the OpenMV camera is used to find the foreground (i.e., the mask) area in the image in the form of color blocks, and relevant information (location information and whether the mask is worn) is fed back.

[0024] S5.4, receiving the information from step S5.3, analyzing and judging the received data, and if the person is wearing a mask, proceed to the next step S6;

[0025] If the pedestrian is not wearing a mask or is wearing a mask abnormally, the pre-processed image recognition information is input to the input end of the stm32 controller;

[0026] S5.5: After the input end of the STM32 controller receives the signal processed by the image processing end of the OpenMV camera, the OpenMV camera generates an identification frame, locks the pedestrian, obtains the relative position of the frame and the screen in real time, and transmits it to the STM32 controller, and the STM32 controller generates a PWM signal;

[0027] S5.6: The automatic tracking device receives the PWM signal from the STM32 controller and, driven by the PWM signal, controls the automatic tracking device to automatically track the person and feeds back the tracking status to the main control system of the automatic tracking device;

[0028] The automatic tracking device includes two states:

[0029] Normal status: people who are not wearing masks are tracked;

[0030] Abnormal status: No person without a mask was tracked;

[0031] S5.7: The automatic tracking device receives the tracking status information transmitted from step S5.6. If the automatic tracking device tracks a person who is not wearing a mask, the process proceeds to the next step S5.8;

[0032] S5.8: The pusher system receives the tracking status information, pushes out a mask through the pusher, and simultaneously issues a voice reminder through the voice reminder module to remind the pedestrian to take the mask and put it on;

[0033] S5.9: If the mask is worn normally after the reminder, go to step S6.

[0034] If the mask is not worn normally, the extracted pedestrian image information will be transmitted to the system cloud;

[0035] Step S6, continue detecting the next frame of image;

[0036] Optionally, step S2 further includes the following steps:

[0037] S2.1. Obtaining the surveillance image captured by the OpenMV camera through the image acquisition unit. The OpenMV camera captures the real-time video stream through its built-in sensor.

[0038] S2.2. Obtain information of the image acquisition unit;

[0039] Through the image processing unit, each frame image in the video stream is format converted into a sampled image;

[0040] Use sliding windows of different sizes to frame a certain image area in the image as a candidate area, and crop the candidate area in the sampled image to obtain a portrait image;

[0041] S2.3. Receive all sampled images, obtain portrait image information, and use the image recognition unit to identify whether there is a face in the sampled image based on facial features, and generate a recognition result;

[0042] S2.4. Transmit the recognition result to the feature detection unit, and use the classifier to identify and judge whether the face is wearing a mask;

[0043] Optionally, a database is constructed for training to improve the accuracy of mask wearing detection. The training methods include:

[0044] (1) Collect a large number of facial images of people wearing and not wearing masks as initial data, and then label and classify these images;

[0045] The face image is input into the trained classifier, and the information output by the learning model is used to indicate whether the target face in the partial face image is wearing a mask, thereby obtaining a face occlusion recognition model;

[0046] (2) Deploy the trained face occlusion recognition model to the OpenMV camera; capture images in real time through the OpenMV camera, extract the mask or facial features in the picture, and then lock and track them, and run the model deployed in it to determine whether the face in the image is wearing a mask.

[0047] A detection method for an image quality distortion detection and recognition system, characterized in that:

[0048] Optionally, step S5.6 includes two states: automatic tracking and manual remote control.

[0049] Optional, visual automatic tracking mode, including:

[0050] ① Use the Openmv camera to identify the image in the lens, locate and identify the face, and determine the probability of wearing a mask (mask) or not wearing a mask (face);

[0051] ② If the pedestrian is determined to be not wearing a mask, the OpenMV camera obtains the center position of the box and compares it with the center of the lens image;

[0052] ③ Obtain the x-axis and y-axis coordinates, and transmit the two position coordinates to the STM32 controller through serial communication;

[0053] ④ According to the x-axis and y-axis coordinates in step ③, the system drives the automatic tracking device so that the first pan-tilt head drives the OpenMV camera to move left and right; at the same time, the system drives the second pan-tilt head so that the second pan-tilt head drives the OpenMV camera to move up and down, thereby tracking people who are not wearing masks.

[0054] Optionally, when the actual situation does not require the tracking mode to be enabled and only a fixed angle needs to be identified and judged, the manual remote control mode is entered; and

[0055] The method for manually adjusting the camera direction includes the following steps:

[0056] 1>Connect and transmit the image observed by the camera to the host computer through the Openmv camera, and display the image in real time on the host computer;

[0057] 2> If the camera's viewing direction does not meet the current monitoring requirements, shake the remote control's joystick to send a radio wave signal

[0058] 3> The first pan-tilt and the second pan-tilt of the automatic tracking device respectively receive the radio wave signal from step 2>, start the first pan-tilt of the automatic tracking device, so that the first pan-tilt drives the OpenMV camera to move left and right; at the same time, start the second pan-tilt, so that the second pan-tilt drives the OpenMV camera to move up and down, thereby adjusting and locking the camera recognition direction.

[0059] Optional, tracking pusher, including:

[0060] Automatic tracking device;

[0061] and a pusher for use with an automatic tracking device; and

[0062] A remote controller used in conjunction with the automatic tracking device, the remote controller comprising a transmitter and a receiver,

[0063] The receiver is connected to the main control system of the automatic tracking device, and

[0064] The main control system of the automatic tracking device is connected to the openmv camera;

[0065] The pusher includes a body, which is detachably connected and

[0066] The body is driven by a drive system, wherein the drive system has a built-in servo for controlling the first pan-tilt head to rotate in the horizontal direction; and

[0067] The first gimbal is provided with a detachably connected body;

[0068] An openmv camera is provided on the body, wherein the openmv camera includes a second pan-tilt platform and a high-performance camera and an stm32 controller installed on the second pan-tilt platform; and

[0069] The second gimbal has a built-in drive system;

[0070] The remote controller also includes a single chip microcomputer,

[0071] The single chip microcomputer comprises:

[0072] NRF24L01 module connection port, used to connect the NRF24L01 module on the remote control, and transmit information and wireless communication using 2.4G signals through the NRF24L01 module on the remote control and the NRF24L01 module set in the stm32 main control system of the body;

[0073] OLED connection port;

[0074] Right joystick wiring port;

[0075] Switch wiring port;

[0076] Left joystick wiring port;

[0077] The NRF24L01 module connection port is connected to the stm32 controller of the main control system of the body through signal transmission;

[0078] The OLED connection port is connected to the OLED display screen.

[0079] The right joystick connection port and the left joystick connection port are connected to the right joystick and left joystick of the remote control respectively.

[0080] The switch wiring port is connected to the control switch of the remote controller.

[0081] Optionally, the machine body includes a push plate, and

[0082] The push plate is provided with a through slot for the mask receiving part to extend into, and

[0083] Furthermore, the mask storage member is divided into two groups of upper and lower cavities by a push plate, and the end surface of the push plate located in the upper cavity is used to place the mask; the mask storage member cooperates with the push plate so that the push plate can move up and down along the longitudinal direction of the body, thereby adjusting the space of the upper cavity of the mask storage member;

[0084] A first stepper motor is provided in the body at a position away from the push plate;

[0085] A through groove is provided on the end surface of the mask storage component away from the first stepper motor;

[0086] The machine body adopts a double-layer shell, and

[0087] The body is provided with an output port.

[0088] The output port is arranged corresponding to the tank body;

[0089] A second stepping motor is installed on the side end surface of the mask storage component near the position of the slot, and

[0090] The driving end of the second stepper motor is connected to the rotating shaft, and

[0091] The rotating shaft passes through the mask storage part,

[0092] Extending into the tank,

[0093] The mask pushing roller is installed on the rotating shaft located in the tank body.

[0094] The mask advancing roller is detachably connected.

[0095] Optionally, the cross-sectional shape of the mask advancing roller is determined by angles α, β and c and the first end face, the second end face and the third end face;

[0096] An extension line of the first end face and an extension line of the second end face intersect to form a first arc-shaped end portion, and the first arc-shaped end portion has an included angle α, 5°≤a≤35°;

[0097] An extension line of the first end face and an extension line of the third end face intersect to form a second arc-shaped end portion, and the second arc-shaped end portion has an included angle c, 12°≤c≤75°;

[0098] The extension line of the second end face intersects with the extension line of the third end face to form a third arc-shaped end portion, and the third arc-shaped end portion has an angle β, 8°≤β≤145°, and the diameter of the third arc-shaped end portion is larger than the first arc-shaped end portion and the second arc-shaped end portion.

[0099] Optionally, an upwardly inclined guide plate is installed on the body at the output port position, and

[0100] The tilt angle R is 3°, and

[0101] A mask separation component is provided on the body at a position opposite to the guide plate, and

[0102] The mask separation component adopts a detachable connection.

[0103] The mask separation component is made of rubber material;

[0104] The distance between the guide plates is L1 through the mask separation assembly, and

[0105] L1 is 0.4mm.

[0106] In summary, the beneficial effects of the present invention are:

[0107] (1) The present invention has two modes: automatic tracking and manual tracking, which greatly improves its applicability and expands its application range. By using the automatic tracking device in conjunction with the OpenMV camera, it can lock on people who are not wearing masks. The present invention can automatically track and warn people who are not wearing masks while recognizing that they are not wearing masks; it has wider applicability.

[0108] (2) The present invention uses the tracking mode to communicate with the serial port of the OpenMV camera and the STM32 microcontroller in actual application. When the OpenMV camera recognizes that there is a person who is not wearing a mask in the lens, it generates an identification frame lock. When the STM32 controller receives the signal processed by the image on the OpenMV camera, it generates a PWM signal and controls the movement of the automatic tracking device under the drive of the PWM signal, thereby increasing the supervision area. The automatic tracking device and the OpenMV camera are used in conjunction to achieve the locking of the person.

[0109] (3) The manual adjustment mode of the present invention converts analog signals into digital signals through the remote control. The analog quantity generated by the resistance voltage change caused by the movement of the remote control joystick is converted into a digital signal through ADC sampling and stored in the single-chip microcomputer. Different PWM signals are then output for different digital quantities to control the movement and adjustment of the automatic tracking device. While improving the applicability, it also prevents the automatic system from failing in an unexpected situation. Through the remote control, the operator can quickly adjust the automatic tracking device to adapt to environmental changes and ensure normal operation.

[0110] When tracking mode isn't necessary and only fixed-angle recognition is needed, the system enters manual remote control mode. Compared to conventional mask recognition systems, this allows the system to adjust the camera's recognition direction in real time via a remote control to accommodate varying monitoring needs.

[0111] (4) The present invention adds a pusher, which can be used in actual applications to identify whether pedestrians are wearing masks through image quality distortion detection and recognition systems. If the pedestrian is not wearing a mask, the person is locked by an automatic tracking device. At the same time, the pusher can provide masks in a timely manner to provide help and support to those who are not wearing masks.

[0112] (5) By adding a mask separation component, when the openmv camera detects that a pedestrian is not wearing a mask, the second stepper motor is started, which drives the rotating shaft to rotate. The rotation of the rotating shaft drives the mask pushing roller set on the rotating shaft to rotate. Since the third arc-shaped end is the largest diameter, when the mask pushing roller rotates to the third arc-shaped end, the third arc-shaped end conflicts with the end face of the mask, and the mask is pushed toward the output port position through the third arc-shaped end until the mask is pushed out of the output port; when the mask is pushed out of the output port, only one mask is allowed to be pushed out, and the second mask will be blocked by the mask separation component, thereby preventing multiple masks from being output at one time.

[0113] (6) The present invention can reduce image distortion and blurring, which can lead to misidentification and misjudgment, improve detection accuracy, and enable accurate detection. When used, threshold segmentation can optimize misidentification and misjudgment caused by complex and changing scene backgrounds, assigning the mask's area to the foreground and other environmental displays to the background. By acquiring the foreground area, the image quality distortion detection and recognition system combined with the relative position after binarization can greatly improve the mask's judgment accuracy and position accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 This is a flowchart of the steps of the mask wearing detection method of the present invention;

[0115] Figure 2 For the present invention Figure 1 A magnified diagram of the middle part of the process;

[0116] Figure 3 For the present invention Figure 1 Another part of the process is enlarged;

[0117] Figure 4 This is a schematic diagram of the single-chip microcomputer structure of the image quality distortion detection and recognition system of the present invention;

[0118] Figure 5 This is a schematic diagram of the overall structure of the pusher for mask wearing recognition and detection of the present invention;

[0119] Figure 6 For the present invention Figure 7 Cross-section at the middle AA position;

[0120] Figure 7This is a schematic diagram of the internal structure of the pusher for mask wearing recognition and detection of the present invention;

[0121] Figure 8 This is a schematic diagram of the overall structure of the pushing mechanism of the pusher for mask wearing recognition and detection of the present invention;

[0122] Figure 9 A schematic cross-sectional view of the mask pushing roller of the pushing mechanism of the pusher for mask wearing identification and detection of the present invention;

[0123] Figure 10 For the present invention Figure 8 A magnified view of the mask separation component structure at the position;

[0124] Figure 11 This is an enlarged view of the push plate structure of the pusher for mask wearing recognition and detection of the present invention;

[0125] Figure 12 This is the acquisition of the foreground area for mask wearing recognition detection in the present invention, and the effect diagram of the mask recognition model combined with the relative position after binarization.

[0126] Reference numerals:

[0127] 100. Tracking pusher;

[0128] 10. Single chip microcomputer;

[0129] 101, OLED connection port;

[0130] 102, NRF24L01 module connection port;

[0131] 103. Right joystick connection port;

[0132] 104. Switch connection port;

[0133] 105. Left joystick connection port;

[0134] 20. Body;

[0135] 201, output port;

[0136] 202, first stepper motor;

[0137] 203, screw;

[0138] 204. Mask storage;

[0139] 205, push plate; 2051, through the tank;

[0140] 207, tank;

[0141] 208, gap;

[0142] 30. Mask separation components;

[0143] 40. OpenMV camera; 401. Second PTZ;

[0144] 50. Drive system;

[0145] 60. Promoting agencies;

[0146] 601, second stepper motor;

[0147] 602, shaft;

[0148] 603, mask pushing roller; 6031, first end surface; 6032, second end surface; 6033, third end surface;

[0149] 70. The First Cloud Platform. DETAILED DESCRIPTION

[0150] In order to make the objectives, technical solutions, and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Although exemplary embodiments are disclosed in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to facilitate a more thorough understanding of the present invention and to fully convey the concept of the present invention to those skilled in the art.

[0151] In the description of this specification, the description with reference to the terms "certain embodiments", "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are contradictory.

[0152] In the present invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "plurality" refers to two or more, unless expressly limited otherwise. Terms such as "installed," "connected," "connected," and "fixed" should be interpreted broadly. For example, "connected" can mean a fixed connection, a detachable connection, or an integral connection; "connected" can mean a direct connection or an indirect connection through an intermediary. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0153] Since the person being tested is in a fast-moving state, the edge sharpness of the image is reduced and the grayscale distinction between pixels is not obvious, which leads to distortion and blurring of the image captured by the camera; this leads to errors in detection and inability to make accurate judgments. In order to solve the above technical problems, Figure 1-12 As shown, this embodiment provides an image quality distortion detection and identification system, including a main system, a main control system of an automatic tracking device, and a pusher system; wherein the main system includes an image acquisition unit, an image processing unit, an image recognition unit, and a feature detection unit.

[0154] like Figure 1-12 As shown, the mask wearing detection and image blur and distortion processing method of this embodiment includes the following steps:

[0155] Step S1: first receive a pedestrian mask wearing detection request sent by the main system;

[0156] Step S2: determine the candidate area (using sliding windows of different sizes to frame a certain image area in the image as the candidate area).

[0157] Use the OpenMV camera to obtain public places and personnel flow monitoring videos to obtain original sample images;

[0158] Step S3, extracting visual features related to the candidate region of the original sampled image (pedestrian's face and mask image);

[0159] Step S4, processing the acquired original sampled image;

[0160] S4.1, use the built-in img.to_grayscale function of the openmv camera to convert the image into a grayscale image, and use the image.statistics.stdev function to obtain the standard deviation of the image grayscale value. Use the grayscale value variance formula to perform a square operation to obtain the grayscale value variance.

[0161] S4.2. Compare the calculated result with the set variance threshold to determine whether the image is blurry. If the image grayscale value variance is higher than the set threshold of 300 (this value is the average value of clear conditions obtained after multiple experiments), the frame is considered to have high clarity and is used, and the process proceeds to the next step.

[0162] S4.3: If the grayscale value variance of the image is lower than the set threshold, the image frame is considered blurred and discarded, and the process goes to step S2 to recapture the image. This effectively prevents misidentification.

[0163] Image gray value variance formula: Where g is the average grayscale value of the image pixels; s is the grayscale value variance of the image pixel grayscale value; is the number of horizontal pixels in the image; The number of vertical pixels in the image; is a pixel, The gray value of are the horizontal and vertical coordinates of the pixel points respectively;

[0164] In this embodiment, the image can be converted to a grayscale image through the built-in img.to_grayscale function of the openmv camera, and the image.statistics.stdev function can obtain the standard deviation of the image grayscale value. Then, the variance can be obtained by performing a square operation using the grayscale value variance formula. When the grayscale value variance of the image is higher than the set threshold, the frame image is considered to have high clarity and is used; otherwise, the frame image is considered to be blurred, the frame rate image is discarded, and the next frame image is collected again to prevent misrecognition and misjudgment.

[0165] Step S5: Use a classifier (SVM model for mask detection) to identify and analyze the extracted data, such as the location coordinates of the pedestrian and whether the pedestrian is wearing a mask.

[0166] S5.1, using the threshold editor of the machine vision system built into the OpenMV camera, obtain the color threshold to be detected. Using the binary image filtering method, for the RGB565 image, adjust the maximum and minimum values of the three parameters L, A, and B in the LAB color space to obtain a specific color threshold list thresholds;

[0167] In this example, the blue mask is used as the training premise. By adjusting the maximum and minimum values of the three parameters L, A, and B in the LAB color space, the threshold parameters of the blue mask in a specific situation are locked. The program uses the sensor.snapshot function to obtain the image information of each frame, and uses the built-in binary function of OpenMV to identify the blue mask landmark and classify it as the foreground; the rest of the image is classified as the background.

[0168] S5.2, identify the mask markers in each frame of the image obtained by the openmv camera, and divide the outer image into foreground and the other images into background; according to the calculation formula,

[0169] If the pixel value falls within the threshold range T, the pixel is classified as foreground (value is 255, indicating white), such as Figure 12 As shown;

[0170] If the pixel value does not fall within the threshold range T, the pixel is classified as background (value 0, indicating black);

[0171] S5.3, after performing binary filtering, use the find_blobs function built into the OpenMV camera to find the foreground (i.e., mask) area in the image in the form of color blocks, and feedback relevant information (location information, and whether the mask is worn);

[0172] S5.4, receiving the information from step S5.3, analyzing and judging the received data, and if the person is wearing a mask, proceed to the next step S6;

[0173] If the pedestrian is not wearing a mask or is wearing a mask abnormally, the pre-processed image recognition information is input to the input end of the stm32 controller;

[0174] The relevant formula is as follows:

[0175]

[0176] in, Indicates that the image is The L, A, and B thresholds at , T represents the range of values that the three parameters L, A, and B can take.

[0177] In this embodiment, threshold segmentation is used to mitigate misidentification and misjudgment caused by complex and changing scene backgrounds. The mask's location is assigned to the foreground, while the rest of the scene is assigned to the background. By capturing the foreground area, detecting image quality distortion, and combining the binarized relative position, the recognition system can significantly improve mask identification accuracy and position precision.

[0178] The terminal output effect of only using the mask recognition and detection system is as follows:

[0179] ***********mask *********

[0180] Model: x110 y150

[0181] The terminal output effect of the recognition system, which can perform both mask recognition and detection and image quality distortion detection and threshold binarization processing, is as follows:

[0182] ***********mask *********

[0183] Model: x110 y150

[0184] Binarization: x105 y147

[0185] It can be seen that in actual effect, the relative position obtained by model recognition is close to that obtained after binarization processing.

[0186] In actual application, the scenes and environments for mask detection are relatively complex, which leads to the complex and changeable background of the image information obtained by the camera, making it easy for the recognition ability to be weakened or the judgment to be wrong when detecting masks.

[0187] This invention employs a binary image filtering method, using the OpenMV camera's threshold editor to obtain the color threshold to be detected, and then performs threshold segmentation. This method can mitigate misidentification and misjudgment caused by complex and changing scene backgrounds, assigning the mask's location to the foreground and other surrounding areas to the background. By acquiring the foreground area and combining the mask recognition model with the binarized relative position, the accuracy of mask identification and positional precision can be greatly improved.

[0188] S5.5: After the input end of the STM32 controller receives the signal processed by the image processing end of the OpenMV camera, the OpenMV camera generates an identification frame, locks the pedestrian, obtains the relative position of the frame and the screen in real time, and transmits it to the STM32 controller, and the STM32 controller generates a PWM signal;

[0189] S5.6: The automatic tracking device receives the PWM signal from the STM32 controller and, driven by the PWM signal, controls the automatic tracking device to automatically track the person and feeds back the tracking status to the main control system of the automatic tracking device;

[0190] In the above embodiment, the automatic tracking device includes two states:

[0191] Normal status: people who are not wearing masks are tracked;

[0192] Abnormal status: No person without a mask was tracked;

[0193] S5.7: The automatic tracking device receives the tracking status information transmitted from step S5.6. If the automatic tracking device tracks a person who is not wearing a mask, the process proceeds to the next step S5.8;

[0194] S5.8: The pusher system receives the tracking status information and pushes out a mask through the pusher. At the same time, the voice reminder module issues a voice reminder (if the pedestrian is not wearing a mask or is wearing the mask improperly), reminding the pedestrian to remove the mask and put it on.

[0195] S5.9: If the mask is worn normally after the reminder, go to step S6.

[0196] If the mask is not worn normally, the extracted pedestrian image information will be transmitted to the system cloud.

[0197] In actual application, when the pedestrian appears in the range of the OpenMV camera lens for the first time, it is successfully recognized and the facial information of the pedestrian is locked. The OpenMV camera immediately saves the successfully recognized photo;

[0198] When the pedestrian is judged to be not wearing a mask and has been out of the range of the openmv camera lens without wearing a mask, it is determined that the pedestrian is not wearing a mask normally.

[0199] At this time, the openmv camera sends the saved picture to the stm32 main control system through the serial port. After receiving the picture information, the main control system transmits the picture to the host computer in the monitoring room via the Internet through the wifi module ESP8266.

[0200] Step S6, continue detecting the next frame of image;

[0201] In this embodiment, the OpenMV camera includes a high-performance camera and an STM32 controller, and has a built-in image sensor, an image processing unit, and a memory, and can directly process image data from the camera without an external computer.

[0202] In this embodiment, step S2 further includes the following steps:

[0203] S2.1. Obtaining the surveillance image captured by the OpenMV camera through the image acquisition unit. The OpenMV camera captures the real-time video stream through its built-in sensor.

[0204] S2.2. Obtain information of the image acquisition unit;

[0205] Through the image processing unit, each frame image in the video stream is format converted into a sampled image;

[0206] Use sliding windows of different sizes to frame a certain image area in the image as a candidate area, and crop the candidate area in the sampled image to obtain a portrait image;

[0207] S2.3. Receive all sampled images, obtain portrait image information, and use the image recognition unit to identify whether there is a face in the sampled image based on facial features, and generate a recognition result;

[0208] S2.4. The recognition result is transmitted to the feature detection unit, and the classifier (mask detection SVM model) is used to determine whether the face is wearing a mask;

[0209] Using a large amount of image data of people wearing and not wearing masks, the model is trained using machine learning or deep learning algorithms (such as convolutional neural networks (CNNs). During the training process, the model learns how to distinguish between images of people wearing masks and those not wearing masks.

[0210] The trained neural network model is deployed to the OpenMV module. When the module captures a new image, it uses this model to classify the image and determine whether the person in the image is wearing a mask.

[0211] During training, in order to improve the accuracy of mask wearing detection, this embodiment constructs a database for training. The training method includes:

[0212] (1) Collect a large number of facial images of people wearing and not wearing masks as initial data, and then label and classify these images;

[0213] (2) Deploy the trained face occlusion recognition model to the OpenMV camera; capture images in real time through the OpenMV camera, extract the mask or facial features in the picture, lock and track them, and run the model deployed in it to determine whether the face in the image is wearing a mask;

[0214] In practical applications, the performance evaluation of mask detection systems focuses on two key metrics: accuracy and false alarm rate. Accuracy refers to the percentage of people the system correctly identifies as wearing and not wearing masks, while false alarm rate refers to the percentage of people the system mistakenly identifies as not wearing masks when wearing them, or vice versa. In practical applications, high accuracy and low false alarm rate are key indicators of system performance.

[0215] In this embodiment, a data set of more than 451 facial images of people wearing and not wearing masks is collected. 80% of them are used as training samples and 20% are used as detection test samples. After multiple training of the recognition model, a training model with an accuracy of nearly 100% and a loss rate close to 0.01 is obtained.

[0216] Furthermore, in this embodiment, it should be noted that the face occlusion (mask) recognition model can be any deep learning model. That is, in addition to the deep learning models listed above, those skilled in the art can use any other applicable deep learning model for face occlusion recognition.

[0217] In this embodiment, using the OpenMV camera for mask wearing recognition is an efficient and convenient solution. It combines the advantages of machine vision and artificial intelligence, can accurately detect faces without masks in complex scenes, and issue warnings, thereby helping to improve public health safety.

[0218] In this embodiment, the present invention uses the tracking mode and, in actual application, utilizes the serial port communication between the OpenMV camera and the STM32 single-chip microcomputer. When the OpenMV camera recognizes that there is a person not wearing a mask in the lens, it generates an identification frame lock. When the STM32 controller receives the signal processed by the OpenMV camera end image, it generates a PWM signal and controls the movement of the automatic tracking device under the drive of the PWM signal, thereby increasing the supervision area. The automatic tracking device and the OpenMV camera are used in conjunction to achieve the locking of the person. The present invention can automatically track and warn the person who is not wearing a mask while recognizing that the person is not wearing a mask; it has wider applicability.

[0219] In this embodiment, step S5.6 also includes two states: automatic tracking and manual remote control; respectively:

[0220] Visual automatic tracking mode, including:

[0221] ① Use the Openmv camera to identify the image in the lens, locate and identify the face, and determine the probability of wearing a mask (mask) or not wearing a mask (face);

[0222] ② If the pedestrian is determined to be not wearing a mask, the OpenMV camera obtains the center position of the box and compares it with the center of the lens image;

[0223] ③ Obtain the x-axis and y-axis coordinates, and transmit the two position coordinates to the STM32 controller through serial communication;

[0224] ④ According to the x-axis and y-axis coordinates in step ③, the system drives the automatic tracking device so that the first pan-tilt head drives the OpenMV camera to move left and right; at the same time, the system drives the second pan-tilt head so that the second pan-tilt head drives the OpenMV camera to move up and down, thereby tracking people who are not wearing masks.

[0225] It includes a main system, a main control system of the automatic tracking device and a pusher system; wherein the main system includes an image acquisition unit, an image processing unit, an image recognition unit and a feature detection unit.

[0226] The present invention can also add a remote controller, which is preferably a 2.4G remote controller. The principle of the remote controller mainly involves the transmission and reception of radio waves, and uses radio signals in the 2.4GHz frequency band to control the remote device.

[0227] Furthermore, the remote control includes a transmitter and a receiver, wherein the transmitter is installed on the handheld remote control end, the receiver is connected to the main control system of the automatic tracking device, and the main control system of the automatic tracking device is connected to the openmv camera, and the openmv camera is installed on the body of the automatic tracking device.

[0228] When the actual situation does not require the use of tracking mode, and only requires identification and judgment of a fixed angle, enter the manual remote control mode; and

[0229] The method for manually adjusting the camera direction includes the following steps:

[0230] 1>Connect and transmit the image observed by the camera to the host computer through the Openmv camera, and display the image in real time on the host computer;

[0231] 2> If the camera's viewing direction does not meet the current monitoring requirements, shake the remote control's joystick to send a radio wave signal

[0232] 3> The first pan-tilt and the second pan-tilt of the automatic tracking device respectively receive the radio wave signal from step 2>, start the first pan-tilt of the automatic tracking device, so that the first pan-tilt drives the OpenMV camera to move left and right; at the same time, start the second pan-tilt, so that the second pan-tilt drives the OpenMV camera to move up and down, thereby adjusting and locking the camera recognition direction.

[0233] Furthermore, the remote control includes a single-chip microcomputer 10, and the single-chip microcomputer 10 is preferably an stm32zet6 single-chip microcomputer, and the single-chip microcomputer 10 is provided with an NRF24L01 module wiring port 102, an OLED wiring port 101, a right joystick wiring port 103, a switch wiring port 104 and a left joystick wiring port 105;

[0234] The NRF24L01 module wiring port 102 is connected to the stm32 controller of the openmv camera, the OLED wiring port 101 is connected to the OLED display, the right joystick wiring port 103 and the left joystick wiring port 105 are respectively connected to the right joystick and left joystick of the remote control, and the switch wiring port 104 is connected to the control switch of the remote control.

[0235] Furthermore, the manual adjustment mode step ④ further includes the steps of:

[0236] (1) Collect ADC signals from the joystick (left joystick / right joystick) through the single-chip microcomputer;

[0237] (2) Shake the joystick to change the resistance value of the microcontroller, thereby changing the resistance voltage at the joystick;

[0238] (3) After the analog quantity is sampled by ADC through the single chip microcomputer, it is converted into digital quantity for storage and processing;

[0239] Convert the acquired digital quantity from 12-bit data to 8-bit data as follows;

[0240] AD2=AD_GetValue(ADC_Channel_2);

[0241] AD3= AD_GetValue(ADC_Channel_3;

[0242] AD0=AD_GetValue(ADC_Channel_0);

[0243] AD2_2=(float)AD2 / 4095*255;

[0244] AD3_2=(float)AD3 / 4095*255;

[0245] AD0_2=(float)AD0 / 4095*255;

[0246] (4) After the transformed digital quantity is collected, the data is processed and calculated in the single chip microcomputer, and the change in the resistance voltage on the joystick (left joystick / right joystick) is further converted into a rotation angle that can be executed by the automatic tracking device.

[0247] (5) After pressing the control switch of the remote control, the internal circuit of the remote control transmitter is activated; the circuit converts electrical energy into a 2.4GHz radio wave signal (the radio wave signal contains the execution command, such as up, down, left, right, start, pause, etc.);

[0248] (6) Radio wave signals are sent through the transmitter antenna of the remote control.

[0249] In the above technical solution, in actual application, the image quality distortion detection and recognition system is used to identify whether the pedestrian is wearing a mask. If the pedestrian is not wearing a mask, the person is locked by the automatic tracking device. However, if the pedestrian does not carry a mask with him / her after being locked, it can only serve as a reminder and cannot provide help and support to those who are not wearing masks. In order to solve the above technical problems, please refer to Figure 7As shown, a tracking pusher 100 for an image quality distortion detection and recognition system includes an automatic tracking device and a pusher used in conjunction with the automatic tracking device. The pusher includes a body 20, which is detachably connected and driven by a drive system 50 to rotate the body 20, thereby driving the OpenMV camera 40 disposed on the body 20 to rotate; this facilitates the OpenMV camera to track pedestrians who are not wearing masks;

[0250] The OpenMV camera includes a second pan-tilt head 401 and a high-performance camera and an STM32 controller installed on the second pan-tilt head 401. The second pan-tilt head 401 has a built-in driving system, which drives the second pan-tilt head to rotate in the vertical direction through the driving system, and then drives the high-performance camera to rotate in the vertical direction through the second pan-tilt head 401.

[0251] Furthermore, the drive system 50 has a built-in servo, and is connected to a first gimbal 70 installed on the drive system 50 through the servo. The built-in servo in the drive system 50 controls the rotation of the first gimbal 70 in the horizontal and vertical directions. A detachably connected body 20 is installed on the first gimbal 70, and disassembling the body 20 makes it convenient for staff to replenish masks immediately.

[0252] The servo and the second gimbal 401 built-in drive system both use DS6230 high-torque servo.

[0253] In this embodiment, the servo and the second gimbal's built-in drive system need to work together to achieve two-dimensional rotation of the first gimbal and the OpenMV camera. The STM32 controller calculates the required rotation angles for the servo and the second gimbal's built-in drive system based on the pre-set rotation angle and direction, and sends corresponding PWM signals to each. The servo and the second gimbal's built-in drive system each receive the required PWM signals and then control the servo and the second gimbal's built-in drive system to rotate according to the predetermined angle, thereby driving the platform to achieve two-dimensional rotation.

[0254] The duty cycle of the two PWM signals is the core content for achieving independent control of the first pan-tilt platform 70 and the second pan-tilt platform 401 in the horizontal and vertical directions; the PWM signal output calculation formula is:

[0255] PWM frequency: Freq = CK_PSC / (PSC+1) / (ARR+1)

[0256] PWM duty cycle: Duty = CCR / (ARR + 1)

[0257] PWM resolution: Reso = 1 / (ARR + 1)

[0258] When ARR is set to 20K and PSC is set to 72, the duty cycle time range of the PWM wave is 0.5ms to 2.5ms according to the above formula. The angle conversion calculation formula of the servo can be obtained from this:

[0259] PWM_SetCompare=Angle / 180*2000+500

[0260] The horizontal rotation angle can be calculated from the above formula to be 10° to 170°. Considering the stability and wear reduction of the structure of the servo, the built-in drive system of the second gimbal 401, and the connection, it is generally set to 10° to 170°.

[0261] The vertical rotation angle range is generally 10° to 170°. Considering the protection of the servo, the built-in drive system of the second gimbal 401 and the stability of the structure of the connection and reducing wear, it is generally set to 45° to 135°.

[0262] In this embodiment, two independent PWM signals from the stm32 controller are received by the built-in drive system of the servo and the second gimbal 401 respectively, and each set of independent PWM signals controls the rotation angle of any one set of servos. By adjusting the duty cycle of the two PWM signals respectively, the first gimbal 70 and the second gimbal 401 can be independently controlled in the horizontal and vertical directions. By independently controlling the horizontal direction of the first gimbal, the first gimbal 70 can drive the openmv camera 40 to adjust the angle from left to right or from right to left, thereby capturing images in different directions. The horizontal rotation angle is 10° to 170°.

[0263] By independently controlling the second pan-tilt platform 401 in the vertical direction, the second pan-tilt platform 401 can drive the openmv camera 40 to adjust the angle upward or downward to obtain images in different directions. The vertical rotation angle range is generally 10° to 170°.

[0264] See also Figure 6-10 As shown, the machine body includes a push plate 205, and a through groove 2051 is formed on the push plate 205 for the mask storage part 204 to extend into.

[0265] Furthermore, the mask storage unit 204 is divided into two groups of upper and lower cavities by the push plate 205, and the end surface of the push plate 205 located in the upper cavity is used to place masks; and the mask storage unit cooperates with the push plate so that the push plate 205 can move up and down along the longitudinal direction of the body 20, thereby adjusting the space of the upper cavity of the mask storage unit 204;

[0266] A first stepper motor 202 is provided in the machine body 20 at a position away from the push plate 205 .

[0267] Furthermore, the driving end of the first stepper motor 202 is connected to the screw 203 through a bearing assembly, and a screw nut is installed on the screw 203, and the screw nut is connected to the push plate 205 through a fixed connection (a fastening screw or a fastening bolt, etc. that can achieve a fixed connection), and the screw nut is located inside the push plate; the screw transmission structure is existing technology and will not be described in detail here.

[0268] In order to prevent the occurrence of a crash of the second stepper motor 601, by setting the threshold value (the maximum rise height of the push plate) to 300mm in advance;

[0269] In this embodiment, when the position detection sensor (not annotated in the figure) installed on the second stepper motor detects that the maximum lifting height of the push plate is 300 mm during use, the system issues a stop command and the motor will no longer execute the rising command. At this time, there are insufficient masks and the staff needs to replenish masks in time.

[0270] In this embodiment, by starting the first stepper motor 202, the first stepper motor drives the lead screw 203 to rotate, thereby driving the push plate to move; thereby compressing the upper cavity to fill the vacancy of the mask output port.

[0271] Furthermore, a through groove 207 is provided on the end surface of the mask storage part 204 away from the first stepper motor 202 to facilitate the mask to be pushed out.

[0272] Furthermore, a second stepper motor 601 is fixedly installed on the side end surface of the mask storage part 204 near the position of the groove body 207 by fastening screws, and the driving end of the second stepper motor 601 is connected to the rotating shaft 602 through a bearing assembly, and the rotating shaft 602 passes through the mask storage part 204 and extends into the groove body 207. A mask pushing roller 603 is installed on the rotating shaft 602 located in the groove body 207, and the mask pushing roller 603 is detachably connected.

[0273] In this embodiment, the second stepper motor 601 is started to drive the rotating shaft 602 to rotate, and the rotation of the rotating shaft 602 drives the mask pushing roller 603 to rotate.

[0274] Furthermore, the second stepping motor 601 , the rotating shaft 602 and the mask pushing roller 603 are combined to form a pushing mechanism 60 .

[0275] Furthermore, the cross-sectional shape of the mask advancing roller 603 is determined by the angles α, β and c and the first end face 6031, the second end face 6032 and the third end face 6033;

[0276] The extension line of the first end surface 6031 and the extension line of the second end surface 6032 intersect to form a first arc-shaped end portion, and the first arc-shaped end portion has an included angle α, 5°≤a≤35°;

[0277] The extension line of the first end surface 6031 and the extension line of the third end surface 6033 intersect to form a second arc-shaped end portion, and the second arc-shaped end portion has an included angle c, 12°≤c≤75°;

[0278] The extension line of the second end face 6032 intersects with the extension line of the third end face 6033 to form a third arc-shaped end portion, and the third arc-shaped end portion has an angle β, 8°≤β≤145°, and the diameter of the third arc-shaped end portion is larger than the first arc-shaped end portion and the second arc-shaped end portion.

[0279] Furthermore, the body 20 is a double-layer shell, and an output port 201 is provided on the body 20 , and the output port 201 is provided corresponding to the slot 207 .

[0280] In this embodiment, when the openmv camera detects that a pedestrian is not wearing a mask, the second stepper motor 601 is started, which in turn drives the rotating shaft 602 to rotate. The rotation of the rotating shaft 602 drives the mask pushing roller 603 set on the rotating shaft 602 to rotate. Since the third arc-shaped end has the largest diameter, when the mask pushing roller 603 rotates to the third arc-shaped end, so that the third arc-shaped end conflicts with the end face of the mask, the mask is pushed toward the position of the output port 201 through the third arc-shaped end until the mask is pushed out of the output port 201.

[0281] In actual use, since the masks are stacked together, when the mask pushing roller 603 is used to push the masks out, two or more masks are often pushed out together, resulting in waste of masks. Figure 12 As shown, an upwardly inclined guide plate is fixedly installed on the body at the position of the output port 201, and the inclination angle R is 3°, and a mask separation assembly 30 is provided on the body 20 at a position opposite to the guide plate, and the mask separation assembly 30 adopts a detachable connection and is made of rubber material;

[0282] The mask separation assembly 30 is used to make the spacing between the guide plates L1, and L1 is 0.4 mm, which is sufficient for outputting only one mask. The thickness of common masks is mostly 0.2-0.3 mm.

[0283] In this embodiment, when the openmv camera detects that a pedestrian is not wearing a mask, the second stepper motor 601 is started, thereby driving the rotating shaft 602 to rotate. The rotation of the rotating shaft 602 drives the mask pushing roller 603 provided on the rotating shaft 602 to rotate. Since the third arc-shaped end has the largest diameter, when the mask pushing roller 603 rotates to the third arc-shaped end, so that the third arc-shaped end conflicts with the end face of the mask, the mask is pushed toward the position of the output port 201 through the third arc-shaped end until the mask is pushed out of the output port 201;

[0284] When the masks are pushed out of the output port 201, only one mask is allowed to be pushed out, and the second mask will be blocked by the mask separation component 30, thereby preventing multiple masks from being pushed out at one time.

[0285] Finally, it should be noted that the above embodiments are intended to illustrate the technical solutions of the present invention only and are not intended to be limiting. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention and are intended to be encompassed by the claims of the present invention.

[0286] Finally, it should be noted that the above embodiments are intended to illustrate the technical solutions of the present invention only and are not intended to be limiting. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention and are intended to be encompassed by the claims of the present invention.

Claims

1. Automatic tracking pusher, characterized by: include: Automatic tracking device; and pushers for use with automatic tracking devices; A remote controller for use with the automatic tracking device; The remote control includes a transmitter and a receiver, wherein the receiver is connected to the main control system of the automatic tracking device, and the main control system of the automatic tracking device is connected to the openmv camera; The pusher includes a body, and the body is driven by a drive system, The openmv camera is arranged on the body. The machine body includes a push plate, The push plate is provided with a through slot for the mask storage part to extend into; The mask storage part cooperates with the push plate; The mask storage unit is divided into two groups of upper and lower cavities by the push plate. The push plate end surface located in the upper cavity is used for placing the mask; A first stepper motor is provided in the body at a position away from the push plate; A through groove is provided on the end surface of the mask storage component away from the first stepper motor; The body is provided with an output port. A second stepping motor is installed on the side end surface of the mask storage component near the position of the slot. The driving end of the second stepper motor is connected to the rotating shaft. The rotating shaft passes through the mask storage part. Extending into the tank, A mask pushing roller is installed on the rotating shaft located in the tank body; A guide plate is installed on the body at the output port. A mask separation component is provided on the body at a position opposite to the guide plate. The distance between the guide plates is L1 through the mask separation assembly, and L1 is 0.4mm.

2. The automatic tracking pusher according to claim 1, characterized in that: The body adopts detachable connection, The machine body adopts a double-layer shell; The driving system has a built-in servo for controlling the first pan / tilt head to rotate in the horizontal direction; The output port is arranged corresponding to the tank body; The mask advancing roller is detachably connected.

3. The automatic tracking pusher according to claim 1, characterized in that: The openmv camera includes a second pan-tilt platform and a high-performance camera and an stm32 controller installed on the second pan-tilt platform; The second gimbal has a built-in drive system; The remote controller also includes a single chip microcomputer, The single chip microcomputer is provided with an NRF24L01 module wiring port, an OLED wiring port, a right joystick wiring port, a switch wiring port and a left joystick wiring port; The NRF24L01 module connection port is connected to the stm32 controller of the main control system of the body for signal transmission; The OLED connection port is connected to the OLED display screen. The right joystick connection port and the left joystick connection port are connected to the right joystick and left joystick of the remote control respectively. The switch wiring port is connected to the control switch of the remote controller.

4. The automatic tracking pusher according to claim 1, characterized in that: The cross-sectional shape of the mask advancing roller is determined by the angles α, β and c and the first end face, the second end face and the third end face; An extension line of the first end face and an extension line of the second end face intersect to form a first arc-shaped end portion, and the first arc-shaped end portion has an included angle α, 5°≤a≤35°; An extension line of the first end face and an extension line of the third end face intersect to form a second arc-shaped end portion, and the second arc-shaped end portion has an included angle c, 12°≤c≤75°; An extension line of the second end face and an extension line of the third end face intersect to form a third arc-shaped end portion, and the third arc-shaped end portion has an included angle β, 8°≤β≤145°, and a diameter of the third arc-shaped end portion is larger than that of the first arc-shaped end portion and the second arc-shaped end portion; The guide plate is tilted upward, and the tilt angle R is 3°; The mask separation component adopts a detachable connection. The mask separation component is made of rubber material.

Images