Underground environment face clock-in method and device based on improved deep residual network

Abstract

The invention discloses an underground environment face clock-in method and device based on an improved deep residual network, and the method comprises the steps: introducing a principal component analysis technology to process a face image before model training, extracting the key features of the face image more comprehensively, selecting RESNET as a pre-weight model of face recognition, carrying out the improvement on the basis of the model, and carrying out the recognition of the face image. An input feature group is separated, and a CBAM attention mechanism is added in convolution transformation before the separation mechanism, so that the network hierarchy is richer and more comprehensive; in the training process, parameters such as the learning rate, the number of iterations, the size of batch processing and an optimization function are continuously iterated, optimized and adjusted to compare and find a model with the optimal accuracy rate, and finally the performance of the model is jointly evaluated by combining a cross entropy loss function and a center point loss function.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for punching in faces in an underground environment based on an improved deep residual network. Background Art

[0002] Coal mine safety is closely related to the accuracy of the coal mine attendance system. Improving the accuracy can not only strengthen the management of the coal mine attendance system, but also protect the life and property safety of the staff. Therefore, the perfection and accuracy of the attendance system plays a vital role in safe production management.

[0003] The coal mine attendance system is based on face recognition algorithms. Face recognition algorithms have gone through three stages: The first stage is the early face recognition algorithm, which is mainly an algorithm based on geometric features. It transforms the face image into a high-dimensional vector in geometric mathematics and projects it into a low-dimensional space to obtain a low-dimensional vector. The second stage of face recognition algorithms mainly adopts the idea of ​​face features + classifiers. The key is the design of artificial features, which can effectively distinguish different people. The third stage is the deep learning recognition stage. After collecting a large number of pictures, they are introduced into the deep network for learning, and then the relevant feature vectors of the pictures are extracted to replace the artificial feature design.

[0004] With the development of high-tech technologies such as deep learning and big data AI intelligence, face recognition technology is becoming more and more mature. However, the development of things is always accompanied by various problems. The coal mine attendance system is facing objective reality problems such as darkness underground and dust on the face. Generally speaking, the attendance process is to capture a facial photo of the attendance personnel and compare the similarity with the corresponding photo of the person previously entered into the system. The "black face" of the attendance personnel is similar to the color difference of the dark environment underground, and it may be impossible to accurately locate the facial contour of the attendance personnel, resulting in incomplete facial image information of the attendance personnel in the picture and inability to accurately compare the similarity of facial photos, which ultimately leads to low image recognition accuracy, directly affecting the attendance personnel's entry and exit check-in. How to accurately identify the facial image of the attendance personnel in the dark underground environment has become an urgent problem to be solved. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, medium and equipment for punching in and out of a mine environment based on an improved deep residual network, aiming to solve the technical problem of accurately identifying facial images of attendance personnel in a dark underground environment.

[0006] To achieve the above objectives, the present application provides a method for face clocking in in an underground environment based on an improved deep residual network, comprising:

[0007] Obtain an underground face video set, and preprocess each face video in the underground face video set to obtain a preprocessed face image set;

[0008] Positioning and aligning a face in each preprocessed face image in a preprocessed face image set to obtain a face image training set, wherein each face image in the face image training set includes a full face image;

[0009] Constructing a training network model, which is an improved deep residual network model, including a separation input feature unit, an intermediate feature transformation unit, and a separation mechanism change unit;

[0010] Each face image in the face image training set is used as the input of the improved deep residual network model, and each input image of the face image is used as the output of the improved deep residual network model to train the improved deep residual network model;

[0011] The underground face image to be clocked in is input into the trained improved deep residual network model, the input image corresponding to the underground face image is obtained, and the face clocking in is completed using the input image.

[0012] Preferably, the method for preprocessing each face video in the acquired underground face video set includes:

[0013] Perform vector transformation on each frame image in each face video to obtain a vector matrix;

[0014] Centralize the obtained vector matrix to obtain a centralized matrix;

[0015] Calculate the covariance matrix of the centered matrix, and obtain the eigenvector matrix from the eigenvalues ​​of the covariance matrix;

[0016] Obtain a target eigenvector matrix from the eigenvector matrix, and project the centralization matrix through the target eigenvector matrix to obtain a processed frame image;

[0017] All the frame images obtained are processed and screened to obtain preprocessed face images.

[0018] Preferably, the input feature separation unit groups the input feature maps according to the channel dimension, and references new parameters to split the grouped feature maps to obtain each split basis array.

[0019] Preferably, the method for the intermediate feature transformation unit to perform intermediate feature transformation includes performing convolution processing on the input feature map, and then adding the CBAM module to each split basis array after the convolution processing to complete the feature transformation.

[0020] Preferably, the method for the separation mechanism changing unit to change the separation mechanism includes:

[0021] For each transformed split basis array, the intermediate representation is fused within the group, and then fused by summing the elements across multiple segments to obtain the combined representation of each basis array, completing the separation mechanism change.

[0022] Preferably, the method for locating and aligning the face in each preprocessed face image in the preprocessed face image set includes:

[0023] Each pre-processed face image is detected and located using the MTCNN algorithm;

[0024] After completing the detection and positioning, the alignment is completed through the DAN algorithm.

[0025] Preferably, the loss of the improved deep residual network model includes:

[0026] L=L b +δL c

[0027] in:

[0028]

[0029]

[0030] Where, L b is the center loss, x i is the category feature, is the feature center of the i-th category, k represents the batch processing size, L c is the cross loss function, y represents the true label value (1 represents a positive value, 0 represents a negative value), is the predicted probability value δ is the balance scalar between the two loss functions.

[0031] To achieve the above objectives, the present application provides an underground environment face punching device based on an improved deep residual network, comprising:

[0032] The acquisition module is used to obtain the underground face video set.

[0033] A processing module is used to preprocess each face video in the acquired downhole face video set to obtain a preprocessed face image set; locate and align the face in each preprocessed face image in the preprocessed face image set to obtain a face image training set, wherein each face image in the face image training set includes a full face image;

[0034] A training network model module, wherein the training network model is an improved deep residual network model, including a separation input feature unit, an intermediate feature transformation unit, and a separation mechanism change unit;

[0035] A training module, using each face image in the face image training set as the input of the improved deep residual network model, and using each input image of the face image as the output of the improved deep residual network model, to train the improved deep residual network model;

[0036] The clocking-in module inputs the underground face image to be clocked in into the trained improved deep residual network model, obtains the input image corresponding to the underground face image, and uses the input image to complete the face clocking-in.

[0037] In order to achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium, characterized in that it includes instructions, which, when running on a computer, enable the computer to execute a face punch-in method in an underground environment based on an improved deep residual network.

[0038] In order to achieve the above-mentioned purpose, the present invention also provides a detection device, which includes: at least one processor, a memory and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute an underground environment face punch-in method based on an improved deep residual network.

[0039] The beneficial effects of this application are:

[0040] Before model training, face images are processed by principal component analysis technology to more comprehensively extract the key features of face images. RESNET is selected as the pre-weighted model for face recognition, and improvements are made on this model by separating the input feature groups and adding the CBAM attention mechanism to the convolution transformation before the separation mechanism, making the network hierarchy richer and more comprehensive. During the training process, the learning rate, number of iterations, batch processing size, optimization function and other parameters are continuously iterated and optimized to compare and find the model with the optimal accuracy. Finally, the cross entropy loss function and the center point loss function are combined to jointly evaluate the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a method for punching in a face in an underground environment based on an improved deep residual network provided by an embodiment of the present invention;

[0042] Figure 2 This is a flow chart of a method for punching in a face in an underground environment in one embodiment of the present invention;

[0043] Figure 3 This is a data collection flow chart in one embodiment of the present invention;

[0044] Figure 4 A structural block diagram of a face punching device in an underground environment based on an improved deep residual network provided in an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the structure of a detection device according to an embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] Figure 1 This is a flow chart of a method for punching in a face in an underground environment based on an improved deep residual network provided by an embodiment of the present invention, refer to Figure 1 The method can be executed by a processor of a computer host or a server. The underground environment face punching method based on an improved deep residual network includes:

[0049] like Figure 1 and Figure 2 A method for punching in a face in an underground environment based on an improved deep residual network is shown, comprising:

[0050] S1. Obtaining a set of underground face videos. The methods used for collecting underground face videos include:

[0051] S11. Installation and selection of acquisition equipment;

[0052] In this embodiment, a fusion substation is used to complete the collection and processing of video data. The camera used to collect underground face video sets in the fusion substation adopts an OV13850 high-definition camera, which is transmitted through a Gigabit Ethernet port and can quickly transmit non-compressed image data in real time. The maximum frame rate can reach 30fps at full resolution. The controller adopts a KXH18C mining intrinsically safe acquisition controller, which is a mining intrinsically safe device and can be used for real-time collection of images of underground personnel. The collected machine vision images are transmitted to the local image processing server for image processing and analysis, providing assistance for intelligent analysis of images.

[0053] The hardware performance of the integrated substation is shown in Table 1:

[0054]

[0055] S12. Data collection;

[0056] The camera provided can be used to detect the faces of people underground. Since the workers are underground, their faces are often in a relatively dark state. Therefore, in this embodiment, the underground workers are represented by "black faces".

[0057] When collecting video images, the camera can be used to collect 30 seconds of video of each person's black face to form a video image for use as a training set for later network training;

[0058] Face image data is obtained through the device--fusion substation. For the specific collection process, see Figure 3 As shown. The specific collection process is: before the collection, the fusion substation will obtain and store the relevant information of the personnel in advance, mainly including the card number, ID card, name, etc. These personnel information are one-to-one. When a person appears in front of the fusion substation screen, the face image will be automatically captured, and the person enters the card number on the substation screen, as shown in Figure 3. It will determine whether the personnel information is accepted and uploaded to the substation. If the system has entered it, the face image will be directly uploaded to the face recognition server for identification and return the classification result; if the system has not entered the face information, the substation will collect the face image by recording the video, and the time is kept at about 30 seconds. The video stream is imported into the OpenCV library for compilation to obtain the face frame image, and finally saved to the corresponding file directory of the face recognition server. The specific calling function is: VideoCapture(File_path), read the video (.mp4 format), read the data according to the object type called: read();

[0059] The face image is uploaded to the face recognition server for recognition, and the returned result will be compared with the face database data. The function of the face database is to compare the image with the acquired data source. A similarity threshold will be set in advance before the comparison. Only when this threshold is reached can the original data be considered to correspond to the face database. The face database mainly stores the face images of the identified persons, one person per folder, and the folder is designated to store the face images of the person;

[0060] S2. Since the accuracy of black faces of underground personnel is low during recognition, each photo in the captured video frame contains a large number of influencing factors, which need to be removed. In this embodiment, in order to ensure that the input photos are more in line with the training face recognition requirements during subsequent training, the collected video images are preprocessed according to the following method;

[0061] S21. Convert video sequence into matrix

[0062] Convert each frame of the obtained video sequence into a vector and arrange these vectors into a matrix in chronological order: Assume that the video sequence contains N frames of images, and the vector of each image is represented by x i , where i is the index of the frame (). Then the operation of converting the video sequence into a matrix can be expressed as:

[0063] X=[x1,x2,x3......x N ]

[0064] Thus, X is a matrix of size M×N, where M is the dimension of each image vector;

[0065] S22. Matrix centralization.

[0066] Subtract the average value of all elements in each column from the element value of each column to get Xcenter:

[0067] mean N =(1 / N)*sum(X,axis=0)

[0068] Among them, mean N is the mean of each column in the X matrix, axis=0 vertically, and N is the number of columns in the X matrix.

[0069] S23. Calculate the covariance matrix and decompose its eigenvalues. The eigenvalues ​​represent the variance of the data set in the direction of the eigenvectors, and the eigenvectors represent the main direction of change of the data set in the corresponding direction;

[0070] C=(1 / N-1)*Xcenter*Xcenter T

[0071] C=V*A*V T

[0072] Where A is the diagonal matrix after eigenvalue decomposition, V is the eigenvector matrix, V T is the transpose of the eigenvector matrix, and Xcenter is the result of centering the S22 matrix.

[0073] S24. Select the principal component. Arrange the eigenvalues ​​in ascending order and select the first K eigenvectors to form the characteristic matrix V = [V1, V2…V k ].

[0074] S25. Projection. Project the original matrix through the eigenvector to obtain a new dataset after reducing the dimension of the original dataset. Project the original matrix Xcenter through the eigenmatrix V to obtain the reduced-dimensional dataset Q:

[0075] Q=V T *Xcenter

[0076] Among them, V is the feature matrix obtained in step S23, and Xcenter is the result of centering the matrix in S22.

[0077] S26. Further enhance the processing of the amplified data

[0078] Call the specific function in the Torchvision.transforms module, and adjust the hue, saturation, rotation angle, scaling, shearing, perspective transformation and other operations of each image in the data set Q to make the enlarged face image data more in line with the basic data training requirements for face recognition. Specifically, first use the random cropping function to crop the input image, use the random rotation function to rotate it, and then use the color transformation function to perform random color transformation. Combined with manual selection, exclude pictures of particularly poor quality and multiple people. It is also necessary to classify the pictures to ensure that the pictures under each work number are complete, reliable and single, so as to increase the richness of the data, thereby improving the model's ability to recognize new data, determine the face images that meet the recognition requirements, and form a pre-processed face image set;

[0079] In this embodiment, data preprocessing is adopted, and its main purpose is to pre-enhance the collected raw data. Due to the complex underground environment and the different postures of personnel, principal component analysis is used to decompose the video sequence, and then the amplitude of the decomposed low-frequency components is amplified to amplify the slight dynamic changes and complete the image preprocessing.

[0080] S3. After the image is preprocessed to obtain a face image set, since the angles of some faces do not meet the requirements, errors that may affect the training process may be caused, so the faces in the face image set need to be adjusted;

[0081] Specifically, the faces in the face image set are detected and aligned. When detecting, the faces are mainly located. The essence of face alignment is to adjust the angle of the face according to the coordinates of the key points. In this embodiment, the key points of the face are key features such as eyes, mouth, and nose.

[0082] S31. In this embodiment, face detection uses the MTCNN algorithm;

[0083] The algorithm mainly uses multiple cascaded network structures to gradually screen and locate faces. Specifically, MTCNN consists of three consecutive networks: Proposal Network (used to generate candidate face regions), Refine Network (further screening and correction of candidate regions generated by P-Net) and Output Network (used for final face detection and key point location). The three networks are used in series;

[0084] S32. After completing the detection of the face in the face image, the angle of the face needs to be adjusted. In one embodiment, the DAN algorithm is used to adjust the face in the face image. The algorithm is a two-stage alignment algorithm. By introducing a key point heat map, the input data entering the convolutional regression network is the entire face, rather than a local image.

[0085] S4. Construct a training network model to recognize the face of the person who punches in later. The training network used in this embodiment is an improved deep residual network model; specifically, the deep learning model algorithms that can complete this embodiment are: Resnet, Alexnet, VGG, Squeezenet, Densenet for model training comparison, and finally select the RESNET model as the preliminary pre-training weight model for face recognition. On this basis, the present invention improves the RESNET model, separates the input feature group, and adds the CBAM attention mechanism in the convolution transformation before the separation mechanism, so as to enrich the model network structure and improve the model performance;

[0086] The deep learning network Resnet structure selected in this embodiment has the following characteristics:

[0087] Solving the deep network degradation problem:

[0088] 1. ResNet solves this problem by introducing residual connections. Residual connections allow information to jump directly, effectively alleviating the gradient vanishing and gradient exploding problems, and enabling the network to be trained deeper and more stable;

[0089] 2. Better feature extraction capabilities;

[0090] 3. Lower network complexity;

[0091] 4. The pre-training effect on large-scale data is good. In summary, the ResNet model is better than other models as a pre-weighted model for face recognition;

[0092] Based on the ResNet model structure, the present invention makes the following improvements:

[0093] S41. Separation of input feature group units

[0094] According to the channel dimension, the input feature map is divided into K groups, each of which is represented as the kth group. These feature maps are used as the input base array. After the determination, a new parameter D is introduced to continue the splitting operation on the base array. D represents the number of divisions within the base array, and each split group is recorded as d. Therefore, according to the setting of the two parameters and the channel dimension, the total number of input feature groups becomes G = KD.

[0095] S42. Intermediate Representation Transformation Unit

[0096] Each individual feature group can perform a separate series of transformations {F1, F2...F G}, then the intermediate representation of each group is S i =F i(x)i∈{1,2…G}. The present invention is mainly represented in the intermediate representation as adding the CBAM module to each split basis array after convolution. Adaptive attention processing is performed on the channel and spatial dimensions of the convolution feature map, and there are mainly two modules: the channel attention module and the spatial attention module. The model's learning ability for key feature points is improved by mixing two attention mechanism modules. The channel attention module first performs an average pooling operation on the input feature map to obtain the average feature value of each channel. Then, the weight of each channel is learned through two fully connected layers, and finally a channel attention vector is obtained. By multiplying it with the input feature map, the feature maps of different channels are weighted. The spatial attention module first performs one-dimensional convolution operations on the input feature map in the horizontal and vertical directions respectively to obtain two attention tensors. Then the two tensors are added and normalized by the sigmoid function to obtain a spatial attention tensor, and the feature maps at different spatial positions are weighted. Finally, the channel attention vector and the spatial attention tensor are multiplied to obtain a weighted feature map. These weighted feature maps can better describe the characteristics of the input image, thereby improving the performance of the model.

[0097] S43. Separation mechanism transformation unit

[0098] After the convolution transformation, the features of the intermediate representation are fused within a single basis array, and then fused by summing the elements across multiple segments to obtain the combined representation of each basis array. The Kth basis array is represented as The specific expression is as follows:

[0099]

[0100] Among them, H, W, C are the sizes of the feature map, specifically the height, width and number of channels of the image, and G = KD.

[0101] After global average pooling, we get the cth component To reduce the dimension of the feature map output by the convolutional layer:

[0102]

[0103] Then, each channel is weighted by the channel soft attention mechanism, and the cth channel The calculation formula is as follows:

[0104]

[0105]

[0106] in, Represents the weight assigned by softmax. Then concatenate each base array to get V:

[0107] V = concat {V 1 ,V 2 ...V K}

[0108] The current input and output are then connected with a standard residual structure and used as the new feature output Q.

[0109] Q=V+θ(X)

[0110] Where θ(x) is the skip connection mapping in the residual block. After the subsequent convolution operation, the final feature map is obtained.

[0111] Before training the deep learning network Resnet, you need to set hyperparameters such as learning rate, batch size, and number of iterations. This article mainly introduces the activation function and loss function in the network training architecture.

[0112] Choosing an activation function

[0113] The network structure used in this algorithm is RESNET, and the activation function used internally is RELU, which mainly introduces nonlinear elements to neurons to increase learning ability. Otherwise, each output is a linear function of the upper layer input. The RELU activation function formula is:

[0114] f(x) = max(0, x)

[0115] Choosing an optimizer and loss function

[0116] The optimization process is actually the process of minimizing the loss function. The smaller the loss function, the better the model performance. The model that uses only one loss function has poor performance. Therefore, the algorithm compares the effects of multiple loss functions, selects multiple appropriate loss functions, assigns weights to them, and jointly measures the quality of the model. The process of continuously adjusting parameters during model training is to minimize the loss function, so as to achieve the optimal model effect. In order to prevent the gradient from disappearing or exploding during the model training process, the L2 regularization term is added to the loss function.

[0117] Similarity comparison: There are two main scenarios for similarity comparison: one is a one-to-one scenario, specifically, judging whether the people in the two images are the same person based on the two given images; the other is to match the most similar face in a face database that collects many face images based on a given image. The algorithm will set a similarity threshold in advance, and only when it is greater than or equal to the threshold will it be considered the same person, and if it is lower than the similarity threshold, it is considered not the same person.

[0118] S5. Train the constructed network. During training, the training set includes input images and output images. The input images are face images that have been positioned and aligned, and the output images are photos of people collected under normal conditions. During the training process, when the network converges, the training is stopped. At this time, the trained network needs to be verified. During the verification, the processed face image is input into the network. If the output face is the face image corresponding to the black face under normal conditions, it means that the network training is successful, otherwise the training continues.

[0119] S6. After the training is completed, the RESNET network model is evaluated. The evaluation method used is as follows: the basic parameters of accuracy are used as the evaluation standard. The loss function is mainly used in the training stage of the model. After each batch of training data is sent to the model, the predicted value is obtained through forward propagation. Then, the difference between the predicted value and the true value is calculated to optimize and adjust the model parameters to minimize the loss value, thereby achieving the purpose of learning.

[0120] The present invention combines the center loss function with the cross entropy loss function to jointly supervise network training. The cross entropy function only considers the distance between different samples, but not the distance between samples of the same type. Therefore, the center loss function makes up for this deficiency. It calculates the distance between samples of the same type and clusters their feature vectors to a central point. It ensures that the aggregation characteristics between samples of the same type are maintained when separating samples of different types. The following is the definition of the center loss function:

[0121]

[0122] Among them, L b is the center loss, x i is the category feature, is the feature center of the i-th category, and k represents the batch processing size. The cross entropy loss function is defined as:

[0123]

[0124] Among them, y represents the true label value (1 represents a positive value and 0 represents a negative value). is the predicted probability value It represents the difference between the true sample label and the predicted probability.

[0125] The loss function of the entire network is:

[0126] L=L b +δL c

[0127] Among them, δ is the balance scalar of the two loss functions. During the training process, the scalar value is continuously optimized to minimize L.

[0128] S7. Clocking in underground. After completing the training, the person who needs to clock in underground will have his / her face photograph collected by the camera. The collected photo will be sent to the RESNET network model. The RESNET network model will be used to generate photos of the person underground in a normal environment. The generated photo will be matched with the photo entered in the system to match the same photo. Since the person's photo and basic information will be entered in the process of photo entry, the basic information of the person can be directly matched after the match is successful, and the work of clocking in underground will be completed.

[0129] Compared with the traditional algorithm, the face punching algorithm trained by the deep convolution separation model in this embodiment has better robustness and accuracy by debugging the training parameters and network architecture. The improved residual network model after training is more accurate and more suitable for underground environment recognition than other training models. In addition, the cross entropy loss function and the center function in the training process are more accurate than the only loss function.

[0130] First, in view of the complex environment underground, the algorithm improves the accuracy of face recognition, which has certain guiding significance for improving the attendance management system for coal mine personnel. Second, it can accurately record the time when staff enter and leave the workplace, effectively monitor personnel with abnormal attendance, which is of vital practical significance for rectifying labor organization, eliminating fraud and improving labor productivity.

[0131] This embodiment provides Figure 4 A face punching device for underground environment based on improved deep residual network is shown, comprising:

[0132] The acquisition module is used to obtain the underground face video set.

[0133] A processing module is used to preprocess each face video in the acquired downhole face video set to obtain a preprocessed face image set; locate and align the face in each preprocessed face image in the preprocessed face image set to obtain a face image training set, wherein each face image in the face image training set includes a full face image;

[0134] A training network model module, wherein the training network model is an improved deep residual network model, including a separation input feature unit, an intermediate feature transformation unit, and a separation mechanism change unit;

[0135] A training module, using each face image in the face image training set as the input of the improved deep residual network model, and using each input image of the face image as the output of the improved deep residual network model, to train the improved deep residual network model;

[0136] The clocking-in module inputs the underground face image to be clocked in into the trained improved deep residual network model, obtains the input image corresponding to the underground face image, and uses the input image to complete the face clocking-in.

[0137] Specifically, when turned on, the acquisition module is used to acquire the black face image of the underground personnel, and send the black face image to the processing module to process the black face image to obtain the processed black face image, and send the black face image to the training network to generate a face image, and the generated face image is matched with the image entered into the system to match the corresponding image and basic information to complete the clocking in of the underground personnel.

[0138] Based on the above embodiments, Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0139] It should be noted that the computer system 400 of the electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0140] Among them, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the underground environment face punching method based on the improved deep residual network in the above embodiment. In RAM 403, various programs and data required for system operation are also stored. CPU 401, ROM 402 and RAM 403 are connected to each other through bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0141] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0142] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.

[0143] It should be noted that the computer-readable medium shown in the embodiment of the present application is used to implement the face punch-in method for underground environment based on improved deep residual network; the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0144] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0146] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0147] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0148] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0149] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0150] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0151] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A face punching method for underground environment based on improved deep residual network, characterized in that: include: Obtain an underground face video set, and preprocess each face video in the underground face video set to obtain a preprocessed face image set; Positioning and aligning a face in each preprocessed face image in a preprocessed face image set to obtain a face image training set, wherein each face image in the face image training set includes a full face image; Constructing a training network model, which is an improved deep residual network model, including a separation input feature unit, an intermediate feature transformation unit, and a separation mechanism change unit; Each face image in the face image training set is used as the input of the improved deep residual network model, and each input image of the face image is used as the output of the improved deep residual network model to train the improved deep residual network model; The underground face image to be clocked in is input into the trained improved deep residual network model, the input image corresponding to the underground face image is obtained, and the face clocking in is completed using the input image.

2. The underground environment face punching method based on improved deep residual network as claimed in claim 1 is characterized in that: The method for preprocessing each face video in the acquired underground face video set includes: Perform vector transformation on each frame image in each face video to obtain a vector matrix; Centralize the obtained vector matrix to obtain a centralized matrix; Calculate the covariance matrix of the centered matrix, and obtain the eigenvector matrix from the eigenvalues ​​of the covariance matrix; Obtain a target eigenvector matrix from the eigenvector matrix, and project the centralization matrix through the target eigenvector matrix to obtain a processed frame image; All the frame images obtained are processed and screened to obtain preprocessed face images.

3. The underground environment face punching method based on improved deep residual network as claimed in claim 1 is characterized in that: The input feature separation unit groups the input feature maps according to the channel dimension, and references new parameters to split the grouped feature maps to obtain each split base array.

4. The underground environment face punching method based on improved deep residual network as described in claim 3 is characterized in that: The method for the intermediate feature transformation unit to perform intermediate feature transformation includes performing convolution processing on the input feature map, and then adding the CBAM module to each split basis array after the convolution processing to complete the feature transformation.

5. The underground environment face punching method based on improved deep residual network as described in claim 4 is characterized in that: The method for the separation mechanism changing unit to change the separation mechanism includes: For each transformed split basis array, the intermediate representation is fused within the group, and then fused by summing the elements across multiple segments to obtain the combined representation of each basis array, completing the separation mechanism change.

6. The underground environment face punching method based on improved deep residual network as claimed in claim 1, characterized in that: The method for locating and aligning a face in each pre-processed face image in the pre-processed face image set includes: Each pre-processed face image is detected and located using the MTCNN algorithm; After completing the detection and positioning, the alignment is completed through the DAN algorithm.

7. The underground environment face punching method based on improved deep residual network as claimed in claim 1 is characterized in that: The losses for improving the deep residual network model include: L=L b +δL c in: Where, L b is the center loss, x i is the category feature, is the feature center of the i-th category, k represents the batch processing size, L c is the cross loss function, y represents the true label value (1 represents a positive value, 0 represents a negative value), is the predicted probability value δ is the balance scalar between the two loss functions.

8. A face punching device for underground environment based on improved deep residual network, characterized in that: include: The acquisition module is used to obtain the underground face video set. A processing module, used for preprocessing each face video in the acquired underground face video set to obtain a preprocessed face image set; Positioning and aligning a face in each preprocessed face image in a preprocessed face image set to obtain a face image training set, wherein each face image in the face image training set includes a full face image; A training network model module, wherein the training network model is an improved deep residual network model, including a separation input feature unit, an intermediate feature transformation unit, and a separation mechanism change unit; A training module, using each face image in the face image training set as the input of the improved deep residual network model, and using each input image of the face image as the output of the improved deep residual network model, to train the improved deep residual network model; The clocking-in module inputs the underground face image to be clocked in into the trained improved deep residual network model, obtains the input image corresponding to the underground face image, and uses the input image to complete the face clocking-in.

9. A computer-readable storage medium, characterized in that: It includes instructions, which, when executed on a computer, enable the computer to execute the underground environment human face punching method based on an improved deep residual network as described in any one of claims 1 to 7.

10. A detection device, characterized in that: The detection equipment comprises: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the underground environment face punching method based on an improved deep residual network according to any one of claims 1 to 7.

Images