Mechanical manufacturing workshop employee safe wearing detection and face recognition method based on deep learning

Abstract

The invention discloses a deep learning-based machine manufacturing workshop employee safe wearing detection and face recognition method. The method comprises the steps of constructing a face sample set; carrying out key point detection according to the face sample set to obtain template coordinates; a real-time image is obtained, the real-time image is detected and positioned through an improved YOLOv8n model, an identification and positioning result is obtained, and the identification and positioning result comprises bounding boxes and corresponding types of human faces and personal protection equipment; cutting the real-time image according to a bounding box of a human face based on a recognition positioning result to obtain a human face image; performing alignment processing on the face image according to the template coordinates; and identifying the aligned face image through an improved FaceNet model to obtain a face vector, and judging the face vector according to the face sample set to obtain personnel identity information so as to realize detection of personal protection equipment and whether foreign personnel exist or not.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target detection and face recognition in workshop supervision, and in particular to a method for detecting safety wear and face recognition of employees in a mechanical manufacturing workshop based on deep learning. Background Art

[0002] The operation of mechanical manufacturing workshops is complex and dangerous, so there are strict requirements for employees' safety wearing and identity authentication. However, the safety wearing of mechanical manufacturing workshops is generally carried out through relevant manual inspections, and its intelligence level is not high. The effect of the existing identity authentication system needs to be improved. At the same time, the above two inspection methods are difficult to combine for detection, and its intelligence level is insufficient. The overall management efficiency needs to be further improved. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a deep learning-based method for detecting and recognizing safety wear of employees in a mechanical manufacturing workshop to solve the problems existing in the above-mentioned prior art.

[0004] To achieve the above objectives, the present invention provides a method for detecting safety wear and face recognition of mechanical manufacturing workshop employees based on deep learning, comprising:

[0005] Construct a face sample set; perform key point detection based on the face sample set to obtain template coordinates;

[0006] Constructing an image target detection model and a face recognition model, wherein the image target detection model adopts an improved YOLOv8n model, and the face recognition model adopts an improved FaceNet model;

[0007] Acquire real-time images of personnel in a mechanical manufacturing workshop, detect and locate the real-time images through an image target detection model, and obtain recognition and positioning results, wherein the recognition and positioning results include bounding boxes and corresponding types of faces and personal protective equipment worn;

[0008] Based on the recognition and positioning result, the real-time image is cropped according to the bounding box of the face to obtain a face image; and the face image is aligned according to the template coordinates;

[0009] The aligned face image is recognized by a face recognition model to obtain a face vector, and the face vector is judged according to the face sample set to obtain the identity information of the person to realize the detection of personal protective equipment and whether the person is an outsider.

[0010] Optionally, the process of building an image object detection model includes:

[0011] Add the MCS module at the output of the backbone network of the YOLOv8n model;

[0012] Introduce the DownSimper module to replace the traditional convolution of the Neck part in YOLOv8n;

[0013] The Shape-IoU loss function is used as the loss function of the YOLOv8n model to construct an image target detection model.

[0014] Optionally, the face recognition model construction process includes:

[0015] Construct a deep convolutional neural network, which includes several convolutional layers, pooling layers and fully connected layers;

[0016] The convolutional layer in the deep convolutional neural network is replaced with the depthwise separable convolutional layer in the MobileNetV1 model, and the SE module is added after the depthwise separable convolutional layer; and the triplet loss function is used as the loss function of the deep convolutional neural network to build a face recognition model.

[0017] Optionally, before building the image object detection model and face recognition model, the following steps are also included:

[0018] Constructing a target sample set, wherein the target sample set includes images containing human faces and personal protective equipment;

[0019] Among them, the image target detection model and the face recognition model are trained by using a ten-fold cross validation method according to the target sample set and the face sample set, and the trained image target detection model and the face recognition model are used.

[0020] Optionally, the process of performing key point detection based on the face sample set includes:

[0021] The MTCNN model is used to detect key points of face samples in the face sample set to obtain template coordinates.

[0022] Optionally, before cropping the real-time image according to the bounding box of the face, the method includes:

[0023] The corresponding category of the personal protective equipment in the identification and positioning results is judged. If the corresponding category of the personal protective equipment is consistent with the preset personal protective equipment, the real-time image is cropped, otherwise a warning is given.

[0024] Optionally, the process of aligning the face image includes:

[0025] Detecting the face image through MTCNN to obtain the coordinates of the face and key points;

[0026] According to the difference between the key point coordinates and the template coordinates, the affine transformation matrix is ​​calculated;

[0027] The facial image is adjusted according to the affine transformation matrix to obtain an aligned facial image.

[0028] Optionally, after aligning the face images, the step further includes:

[0029] Performing image equalization processing on the aligned face image, and using the image equalization processing as input data of the face recognition model to obtain a face vector.

[0030] Optionally, the process of judging the face vector includes:

[0031] Obtain a sample face vector of a face sample in a face sample set; perform cosine similarity calculation between the face vector and the sample face vector, and perform a threshold judgment on the cosine similarity calculation result. If the cosine similarity calculation result is less than the threshold, the employee information corresponding to the face sample is obtained as the personnel identity information; otherwise, the personnel identity information is that of an outsider.

[0032] Optionally, after obtaining the personnel identity information, the following is also included:

[0033] If the person's identity information is that of an outsider, a warning and reminder will be issued. If the person's identity is the employee information corresponding to the face sample, the corresponding type of personal protective equipment worn by the person identified in the real-time image will be recorded.

[0034] Compared with the prior art, the present invention has the following advantages and technical effects:

[0035] The present invention realizes comprehensive monitoring of employee safety protection status and effective confirmation of identity by integrating image target detection and face recognition technology. The method has the advantages of high efficiency, accuracy, reliability, etc., and has broad application prospects in the machinery manufacturing industry. Specifically, in order to improve the effect of safety management, YOLOv8n and FaceNet are combined and applied to the workshop to achieve more intelligent safety management. For safety wearing detection, the YOLOv8n model has been improved in a targeted manner, and the MCS module has been integrated to improve the ability to capture detailed features, and the DownSimper module has been introduced to replace the traditional convolution CONV of the Neck part in YOLOv8n to improve the performance of feature fusion and downsampling. At the same time, the Shape-IoU loss function is used to improve the regression accuracy of the bounding box. These improvements enable the model to more accurately identify whether employees are wearing necessary safety equipment in a complex workshop environment. FaceNet is responsible for verifying the identity of employees. Such a system not only improves the efficiency and accuracy of safety monitoring, but also enhances the intelligent level of workshop management. By combining these two technologies, a comprehensive safety and identity management system can be implemented in the workshop. The improved YOLOv8n model can automatically detect whether employees are wearing safety masks, while FaceNet provides effective identity authentication. The combination of the two improves security, reduces the need for manual inspections, and improves overall management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0037] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0038] Figure 2 4 is a structural diagram of the DownSimper module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] The present invention discloses a method for detecting and recognizing safe wear of employees in a mechanical manufacturing workshop based on deep learning, and belongs to the technical field of image target detection and face recognition. In the technical scheme related to the present invention, images containing clear faces and at least one personal protective equipment are first collected to form a person_dataset data set. A high-definition camera is used to capture clear frontal images of employees in a specified area. At the same time, combined with the employee entry information, a face database containing diverse facial features is established, recorded as face_dataset. MTCNN is used to perform key point detection on each sample in the face_dataset database, and the coordinates of these key points are recorded as template coordinates for subsequent face alignment. Due to the YOLOv8n model's capture of detail features and the accuracy of the bounding box, as well as background interference problems, especially in highly complex and dynamic environments. Therefore, the YOLOv8n model is targetedly improved. The MEW multi-axis external weight module is combined with the CBAM attention, and the MCS module is proposed. After being placed in the backbone network, it can better capture detail features, thereby improving detection accuracy and reducing background interference. After that, the DownSimper module is introduced to replace the traditional convolution of the Neck part in YOLOv8n to further improve the performance of feature fusion and downsampling. Finally, the Shape-IoU loss function is used, which calculates the loss by focusing on the shape and scale of the bounding box itself, making the bounding box regression more accurate. The depth-separable convolution used by MobileNetV1 in FaceNet may lead to reduced feature expression ability, so the SE (Squeeze-and-Excitation) module is added to the network to enhance the responsiveness to important features, thereby improving the representation ability of the model. The improved YOLOv8n model and face recognition model are trained by ten-fold cross validation using the existing person_dataset and face_dataset datasets. Then, the trained workshop employee wear detection model is used to infer and recognize the target image. In the recognition result, the bounding box of the face category is screened out, and the face image is cropped out according to the bounding box of the face category in the original image, and the MTCNN is applied to detect its face and key point coordinates. Then, the affine transformation matrix is ​​calculated based on the difference between the detected key points and the template coordinates. Through this transformation matrix, the face image is adjusted to a uniform posture and scale. After the face is aligned, image equalization technology is further used to optimize the image quality. The processed image is input into the FaceNet model. If the recognition result shows that the employee is an outsider, the system will immediately issue an outsider warning to remind the safety manager to pay attention and take corresponding measures. If the recognition result is a workshop employee, the system confirms his identity and records his compliance with various protective equipment.This process design of first performing safety wearing detection and then facial recognition can not only monitor the safety wearing status of employees in real time, but also effectively manage the entry and exit of workshop personnel and improve the overall safety level of the workshop.

[0042] like Figure 1 As shown, the present invention is mainly verified by experimental and actual measurement methods. The implementation steps are as follows:

[0043] S1. Dataset collection and annotation:

[0044] Collect images of people with clear faces and at least one type of personal protective equipment to form the person_dataset dataset. Use a high-definition camera to capture clear frontal images of employees in the designated area. At the same time, combined with the employee entry information, establish a face database containing diverse facial features, recorded as face_dataset. Use MTCNN to detect key points for each sample in the face_dataset database, and record the coordinates of these key points as template coordinates for subsequent face alignment.

[0045] S2. Model improvement and training:

[0046] The YOLOv8n model was optimized to address the feature complexity and background interference issues in the detection of safety protection products. At the same time, in the FaceNet model, the depth-separable convolution of MobileNetV1 is used. Although MobileNetV1 reduces the amount of calculation and model complexity through depth-separable convolution, it may reduce the feature expression ability. Therefore, the SE (Squeeze-and-Excitation) module is added after the convolution layer to adaptively adjust the channel features and enhance the responsiveness to important features, thereby improving the feature differentiation ability while maintaining computational efficiency. The improved YOLOv8n model and face recognition model are both trained using the existing person_dataset and face_dataset datasets through ten-fold cross validation.

[0047] S3, safe wearing detection and face cropping:

[0048] Use the trained improved YOLOv8n model to infer the real-time image. If an employee is found not wearing any protective equipment, the system will immediately trigger a missing wear warning to remind the management to take timely measures. If the employee has worn complete safety protection equipment, the system will continue to perform the face recognition process through the FaceNet model. Filter out the bounding boxes belonging to the face category from the detection results, and crop the face image from the original image based on these bounding boxes to prepare for subsequent face recognition.

[0049] S4. Face alignment:

[0050] After obtaining the cropped face image in step S3, MTCNN is first applied to detect the face and key point coordinates of the face image. Then, the affine transformation matrix is ​​calculated based on the difference between the detected key points and the template coordinates in step S1. Through this transformation matrix, all collected face images are adjusted to a uniform posture and scale, realizing face alignment processing and improving the accuracy of subsequent face recognition.

[0051] S5. Image equalization:

[0052] Image equalization technology is used to perform image equalization on the aligned face images. Image equalization mainly adjusts the brightness and contrast distribution of the image to make the image quality more stable and reduce the impact of light changes on the recognition effect.

[0053] S6. Face recognition and safety wearing compliance judgment:

[0054] The image after image equalization is input into the FaceNet model for face recognition. If the recognition result shows that the person is an outsider, an outsider warning is triggered immediately to remind the safety management personnel to pay attention and take corresponding measures. If the recognition result shows that the person is a workshop employee, he / she is allowed to enter the workshop to work, and the system records his / her identity and the wearing of various protective equipment.

[0055] The above technical solution is described in detail:

[0056] As some embodiments, step S1 includes the following contents:

[0057] Collect images of faces, helmets, safety vests, and safety gloves to form the person_dataset dataset. Use a high-definition camera to capture clear frontal images of employees in a designated area. At the same time, combined with employee entry information, establish a face database containing diverse facial features, denoted as face_dataset. Use MTCNN to detect key points for each sample in the face_dataset database, and record the coordinates of these key points as template coordinates for subsequent face alignment.

[0058] Specifically, step S1 includes the following specific steps:

[0059] S11: Go to the mechanical manufacturing workshop for field shooting and collect images in multiple time periods (day and night) to ensure that the collected work scenes are diverse and cover various work situations. After communicating with the employees and obtaining their consent, shoot under different lighting conditions, angles and distances. Filter out images containing clear faces and at least one type of personal protective equipment from the collected images, try to ensure that the images contain different types and styles of various protective equipment, and collect at least thousands of images to ensure the generalization ability of the model. Use LabelImg to annotate an image dataset containing four categories of face, safety helmet, safety vest and safety gloves, recorded as person_dataset dataset.

[0060] S12: Use a high-definition camera to capture clear frontal images of employees in the designated area, resize the images to a fixed size (160x160 pixels), and normalize the image brightness and color. At the same time, combined with the employee entry information, establish a face database containing diverse facial features, recorded as face_dataset. Use MTCNN to detect key points for each sample in the face_dataset database, and record the coordinates of these key points as template coordinates for subsequent face alignment.

[0061] Wherein step S2 comprises the following contents:

[0062] In step S2, the YOLOv8n model and FaceNet model are improved specifically. Then, the improved YOLOv8n model and face recognition model are trained respectively by the established person_dataset and face_dataset. In the mechanical manufacturing scenario, the safety wearing detection of workers is crucial. However, the YOLOv8n model has certain deficiencies in capturing detailed features and the accuracy of bounding boxes, as well as background interference problems, especially in highly complex and dynamic environments. In view of the feature complexity and background interference problems in the detection of safety protective equipment, three key improvements are proposed for YOLOv8n: the MCS module based on multi-channel attention and multi-dimensional weighting, the DownSimper module, and the Shape-IoU loss function.

[0063] Specifically, the MCS module combines the MEW multi-axis external weight module with the CBAM attention. The MCS module effectively extracts global and local information through multi-dimensional weighting, so as to better capture the subtle features of safety equipment. The CBAM mechanism makes precise adjustments and optimizations on this basis, so that the model can focus more on the wearer of the safety equipment, significantly reduce background interference, and improve detection accuracy. After inserting the MCS module into the backbone network of YOLOv8n, that is, setting the MCS module at the output end of the backbone network, and taking the features processed by the MCS module as the final output, the algorithm can better focus on the features of key targets such as faces, helmets, safety vests and safety gloves. The formula for the specific process of MEW is as follows:

[0064] x1,x2,x3,x4=Split(X)

[0065]

[0066] x'4=DW(x4)

[0067] Y=Concat(x1',x'2,x'3,x'4)+X

[0068] Where i=1, 2, 3 correspond to the first three branches respectively. (I,J) and Represent the learnable external weights and two-dimensional discrete Fourier transform of the corresponding axis respectively. ⊙ represents element-by-element multiplication. When i=1, (I, J) represents the Height-Width axis; when i=2, (I, J) represents the Channel-Width axis; when i=3, (I, J) represents the Channel-Height axis. Denotes a two-dimensional inverse discrete Fourier transform. DW denotes a depthwise separable convolution.

[0069] The convolution layer in the Neck part of YOLOv8n is prone to information loss and fusion limitations in feature fusion and downsampling. The DownSimper module is introduced to replace the traditional convolution layer CONV in the Neck part. Figure 2As shown in the figure, the module divides the input feature map into two main branches. The first branch downsamples through 3x3 convolution with a stride of 2 to extract deeper features; the second branch uses 1x1 convolution to keep the spatial size unchanged and adjust the number of channels to ensure the consistency of feature dimensions. In the second branch, the feature map after convolution is combined with parallel maximum pooling and average pooling operations to extract features from different angles and enhance the diversity and robustness of features. In the end, the module forms three independent paths: one from the downsampled 3x3 convolution, and the other two from the maximum pooling and average pooling after the 1x1 convolution. By splicing the feature maps of these three paths in the channel dimension, rich output features are generated. This design significantly improves the detection accuracy and robustness of the model, enabling it to accurately identify safety protection equipment in complex backgrounds and different postures.

[0070] In order to improve the accuracy of bounding box regression, the Shape-IoU loss function is used. Compared with the traditional IoU loss function, Shape-IoU not only considers the degree of overlap between bounding boxes, but also pays attention to the shape and scale of the bounding box itself. This design enables the loss function to more accurately reflect the differences between bounding boxes, thereby guiding the model to perform more accurate bounding box regression. By introducing the Shape-IoU loss function, the performance of YOLOv8n in tasks such as helmet detection is further improved. The Shape-IoU loss function is defined as follows:

[0071]

[0072] Among them, IoU is used to measure the overlap between the predicted box and the real box, where B and B gt Represents the predicted box and the true box respectively. The IoU value is the area of ​​the intersection of the predicted box and the true box divided by the area of ​​their union. The larger the value, the higher the overlap between the predicted box and the true box, and the more accurate the prediction.

[0073]

[0074] Among them, w scale is the width adjustment formula, h scale is the height adjustment formula. scale is the scale factor, w scale and h scale They are the weight coefficients in the horizontal and vertical directions respectively. Their values ​​are related to the shape of the real frame and are used for scale adaptation of the model.

[0075]

[0076] Among them, distance shapeIt is the weighted Euclidean distance between the center point of the predicted box and the center point of the real box. The weight is the adjusted width and height, reflecting the influence of the distance between the center points on shape matching. c Represents the coordinate of the center point of the prediction box on the x-axis. c gt is the coordinate of the center point of the ground-truth box on the x-axis. c is a constant used to normalize the distance so that the distance calculation between boxes of different scales is consistent.

[0077]

[0078] Among them, Ω shape is the shape loss function, which sums the width (w) and height (h) scales. θ represents the degree of attention paid to shape loss and is usually set to 4.

[0079]

[0080] Among them, w w and w h are the adjusted width and height, w gt and h gt Represents the width and height of the real box, which also takes into account the maximum value of the predicted box and the real box to ensure that the adjusted box can cover the actual target area.

[0081] L Shape-IoU =1-IoU+distance shape +0.5×Ω shape

[0082] Finally, the Shape-IoU method improves the accuracy of bounding box regression by introducing additional attention to shape and scale. It combines the intersection over union (IoU) with shape and scale adjustment factors to more accurately capture the actual shape and size of the object when processing the bounding box.

[0083] FaceNet uses deep convolutional neural networks (CNNs) to achieve efficient face recognition by mapping face images to a high-dimensional embedding space. The process includes multiple convolutional layers, pooling layers, and fully connected layers, which work together to extract detailed features in the image. In order to improve computational efficiency and reduce model complexity, FaceNet uses the deep separable convolution in MobileNetV1. This convolution method significantly reduces the amount of computation and model complexity by decomposing the standard convolution into pixel-by-pixel convolution and channel-by-channel convolution. However, this simplification may impair the expressiveness of features, resulting in a reduction in feature expressiveness. To solve this problem, the SE (Squeeze-and-Excitation) module is introduced in FaceNet, and the SE (Squeeze-and-Excitation) module is added to the deep separable convolution. The SE module enhances the response to key features by adaptively adjusting the weights of each channel, thereby improving the model's ability to capture details and enhancing the model's representation ability. Finally, by adding the SE module after the depthwise separable convolution to generate the embedding vector, the facial images of the same person are close to each other in the embedding space, while the images of different people are separated in the space, thereby achieving high-precision face recognition. Through the above-mentioned improvement method, the network can improve the ability to distinguish features while maintaining computational efficiency, further improving the accuracy and robustness of facial recognition.

[0084] FaceNet uses a triplet loss function during training to directly optimize the embedding vector so that images of the same person are closer in the embedding space, while images of different people are more separated. Specifically, a triplet includes a reference sample (Anchor), a positive sample (Positive), and a negative sample (Negative). The loss function aims to minimize the distance between the reference sample and the positive sample, while maximizing the distance between the reference sample and the negative sample. This loss function design enables FaceNet to perform well in face recognition tasks and significantly improves recognition accuracy.

[0085] The triplet loss function formula is as follows:

[0086]

[0087] in and are the feature vectors of anchor points, positive samples and negative samples respectively, α is the threshold, [z] + It means that loss occurs when the difference between the intra-class distance and the inter-class distance is greater than α.

[0088] Finally, ten-fold cross validation was used to perform ten-fold cross validation on the improved YOLOv8n and FaceNet models respectively. The dataset was randomly divided into 10 subsets, 9 subsets were used as training sets each time, and the remaining subset was used as a validation set. This was repeated 10 times to ensure that each subset was used as a validation set once.

[0089] Wherein step S3 comprises the following contents:

[0090] S3: Use the trained YOLOv8n improved model to quickly scan the real-time image, detect and locate targets such as faces, safety helmets, safety vests, and safety gloves in the image. And determine whether the employees are wearing complete safety protection equipment. If it is found that an employee is not wearing any protective equipment, the system will immediately trigger a missing wearing warning to remind the management to take timely measures. If the employee is wearing complete safety protection equipment, the system will continue to execute the face recognition process. In the detected results, filter out the bounding boxes belonging to the face category, which are the face areas. Crop the face image from the original image based on the bounding box of the face area.

[0091] Wherein step S4 comprises the following contents:

[0092] S4: After obtaining the cropped face image, MTCNN is first used to detect the face and key point coordinates. Based on the difference between the detected key points and the template coordinates, the affine transformation matrix is ​​calculated. This matrix describes how to rotate, scale, and translate the current face image so that its key points are accurately aligned with the template coordinates. By applying this transformation matrix, all collected face images are adjusted to a uniform posture and scale.

[0093] Specifically, after obtaining the cropped image in step S3, MTCNN is first applied to detect the face position and key point coordinates. Then, the affine transformation matrix is ​​calculated based on the difference between the detected key point coordinates and the template coordinates in step S1. This matrix describes how to accurately align the key points of the current face image with the template coordinates through operations such as rotation, scaling, and translation. By applying this transformation matrix, all collected face images are adjusted to a uniform posture and scale, thereby improving the stability and accuracy of the face recognition algorithm. MTCNN is a lightweight cascade convolutional neural network that gradually refines the face candidate frame through three network stages of different depths (P-Net, R-Net, O-Net), and finally outputs the face position and the coordinates of five key points (center of both eyes, tip of the nose, left and right sides of the mouth corners).

[0094] Wherein step S5 comprises the following contents:

[0095] S5: Considering that the light in the machine manufacturing workshop changes frequently and unevenly, direct face recognition may cause the recognition rate to drop due to the difference in light. After face alignment, image equalization technology is further used to optimize the image quality. Image equalization mainly adjusts the brightness and contrast distribution of the image to make the pixel value distribution more uniform across the entire image range, thereby reducing the impact of light changes on the recognition effect. Therefore, adaptive histogram equalization is used to effectively deal with the problem of local light changes.

[0096] Wherein step S6 comprises the following steps:

[0097] S61: Input the processed face image into the FaceNet model to extract the feature vector (embedding) of the face. Calculate the cosine similarity between the feature vector of the face to be verified and the feature vector of the face in the face database face_dataset, and determine whether it is a known identity based on the similarity. The value range of cosine similarity is between 0 and 1. The closer the value is to 1, the more similar the two vectors are, and the closer the value is to 0, the less similar they are. Set a suitable cosine similarity threshold based on the needs of the actual application and the performance of the system. If the cosine similarity between the new embedding and an embedding in the database is higher than this threshold, they are considered to belong to the same person; otherwise, they are considered to not belong to the same person. The formula for cosine similarity is as follows:

[0098]

[0099] Where A and B represent the feature vectors of two face images in the vector space respectively.

[0100] S62: If the identification result shows that the employee is an outsider, the system will immediately issue an outsider warning to remind the safety manager to pay attention and take corresponding measures. If the identification result shows that the employee is a workshop employee, he / she will be allowed to work in the workshop, and the system will record his / her identity and the compliance of the various protective equipment he / she wears. In this way, we can not only monitor the safety wearing of employees in real time, but also effectively manage the entry and exit of workshop personnel and improve the overall safety level of the workshop.

[0101] The present invention realizes comprehensive monitoring of employee safety protection status and effective identification of identity by integrating image target detection and face recognition technology. The method has the advantages of high efficiency, accuracy and reliability, and has broad application prospects in the machinery manufacturing industry.

[0102] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting safety wear and face recognition of mechanical manufacturing workshop employees based on deep learning, characterized in that: include: Construct a face sample set; Perform key point detection based on the face sample set to obtain template coordinates; Constructing an image target detection model and a face recognition model, wherein the image target detection model adopts an improved YOLOv8n model, and the face recognition model adopts an improved FaceNet model; Acquire real-time images of personnel in a mechanical manufacturing workshop, detect and locate the real-time images through an image target detection model, and obtain recognition and positioning results, wherein the recognition and positioning results include bounding boxes and corresponding types of faces and personal protective equipment worn; Based on the recognition and positioning result, the real-time image is cropped according to the bounding box of the face to obtain a face image; and the face image is aligned according to the template coordinates; The aligned face image is recognized by a face recognition model to obtain a face vector, and the face vector is judged according to the face sample set to obtain the identity information of the person to realize the detection of personal protective equipment and whether the person is an outsider.

2. The method according to claim 1, characterized in that The process of building an image object detection model includes: Add the MCS module at the output of the backbone network of the YOLOv8n model; Introduce the DownSimper module to replace the traditional convolution of the Neck part in YOLOv8n; The Shape-IoU loss function is used as the loss function of the YOLOv8n model to construct an image target detection model.

3. The method according to claim 1, characterized in that The process of building a face recognition model includes: Construct a deep convolutional neural network, which includes several convolutional layers, pooling layers and fully connected layers; The convolutional layer in the deep convolutional neural network is replaced with the depthwise separable convolutional layer in the MobileNetV1 model, and the SE module is added after the depthwise separable convolutional layer; and the triplet loss function is used as the loss function of the deep convolutional neural network to build a face recognition model.

4. The method according to claim 1, characterized in that: Before building the image target detection model and face recognition model, it also includes: Constructing a target sample set, wherein the target sample set includes images containing human faces and personal protective equipment; Among them, the image target detection model and the face recognition model are trained by using a ten-fold cross validation method according to the target sample set and the face sample set, and the trained image target detection model and the face recognition model are used.

5. The method according to claim 1, characterized in that The process of key point detection based on the face sample set includes: The MTCNN model is used to detect key points of face samples in the face sample set to obtain template coordinates.

6. The method according to claim 1, characterized in that The real-time image is cropped according to the bounding box of the face, including: The corresponding category of the personal protective equipment in the identification and positioning results is judged. If the corresponding category of the personal protective equipment is consistent with the preset personal protective equipment, the real-time image is cropped, otherwise a warning is given.

7. The method according to claim 1, characterized in that The process of aligning the face image includes: Detecting the face image through MTCNN to obtain the coordinates of the face and key points; According to the difference between the key point coordinates and the template coordinates, the affine transformation matrix is ​​calculated; The facial image is adjusted according to the affine transformation matrix to obtain an aligned facial image.

8. The method according to claim 1, characterized in that After the face image is aligned, the method further includes: Performing image equalization processing on the aligned face image, and using the image equalization processing as input data of the face recognition model to obtain a face vector.

9. The method according to claim 1, characterized in that: The process of judging the face vector includes: Obtain a sample face vector of a face sample in a face sample set; perform cosine similarity calculation between the face vector and the sample face vector, and perform a threshold judgment on the cosine similarity calculation result. If the cosine similarity calculation result is less than the threshold, the employee information corresponding to the face sample is obtained as the personnel identity information; otherwise, the personnel identity information is that of an outsider.

10. The method according to claim 9, characterized in that After obtaining the personnel identity information, it also includes: If the person's identity information is that of an outsider, a warning and reminder will be issued. If the person's identity is the employee information corresponding to the face sample, the corresponding type of personal protective equipment worn by the person identified in the real-time image will be recorded.

Images