A multi-face detection method, device and storage medium

Abstract

The present invention discloses a multi-face detection method, which belongs to the field of computer vision technology, and solves the technical problem that the detection speed and accuracy of the traditional image processing method cannot meet the requirements. The method includes step S1. constructing a face image data set; step S2. constructing a multi-face detection neural network; step S3. iteratively training the multi-face detection neural network according to the face image data set to obtain a multi-face detection model; step S4. constructing a multi-face detection decoder; step S5. inputting the image to be detected into the multi-face detection model to obtain a prediction result, and inputting the prediction result into the multi-face detection decoder to obtain a detection result. The present invention also discloses a computer device and a computer-readable storage medium. The present invention makes a series of improvements on the basis of the convolutional neural network, including introducing a dimensionality reduction convolution module and a multi-scale convolution module. Compared with the existing face detection technology, it not only improves the detection speed, but also improves the accuracy of face detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically, to a multi-face detection method, device and storage medium. Background Art

[0002] At present, face detection technology has been widely used in daily and work life. It not only plays an important role in the fields of public security, economic security and military security, but also brings great convenience to people in daily life. It has very important research value.

[0003] In related technologies, traditional image processing technology is used to perform face detection on images, such as the Adaboost-based face detection algorithm. The core idea is to automatically select several classifiers from the space of multiple weak classifiers to form a strong classifier with strong classification ability, thereby making a predictive classification on whether there is a face in an image.

[0004] The above method is easily affected by complex environments. In multi-face images with complex backgrounds, false detection or missed detection is likely to occur. At the same time, for high-quality images in today's big data context, its detection speed and accuracy do not meet the requirements and have great limitations. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the above-mentioned deficiencies in the prior art. One purpose of the present invention is to provide a multi-face detection method that can improve detection speed and accuracy.

[0006] A second objective of the present invention is to provide a computer device.

[0007] A third objective of the present invention is to provide a computer-readable storage medium.

[0008] In order to achieve the above objective 1, the present invention provides a multi-face detection method, comprising the following steps:

[0009] Step S1. Constructing a face image dataset;

[0010] Step S2. Constructing a multi-face detection neural network;

[0011] Step S3. iteratively training the multi-face detection neural network according to the face image data set to obtain a multi-face detection model;

[0012] Step S4. Construct a multi-face detection decoder;

[0013] Step S5. Input the image to be detected into the multi-face detection model to obtain a prediction result, and input the prediction result into the multi-face detection decoder to obtain a detection result;

[0014] Wherein, step S2 comprises:

[0015] Step S21. Construct a dimensionality reduction convolution module, which is located at the beginning of the network and is used to process the input image of the input layer. The dimensionality reduction convolution module includes a Conv layer, an Activation layer, a BN layer, and a MaxPooling2D layer;

[0016] The Conv layer contains 12 5×5 convolution kernels, the convolution step is set to 4, the padding value is set to the same, and the convolution kernel parameters are initialized using Gaussian random numbers with a mean of 0 and a variance of 1;

[0017] The Activation layer is connected after the Conv layer, and Relu is used as the activation function;

[0018] The BN layer is connected after the Activation layer, and the values ​​of gamma_regularizer and beta_regularrizer are both set to 0.0001;

[0019] The MaxPooling2D is connected after the BN layer, the pooling window size is set to 2×2, the pooling step size is set to 2, and the padding is set to the same;

[0020] Step S22. Construct a multi-scale convolution module: the multi-scale convolution module includes a Conv_1 layer, a Conv_2 layer, a Conv_3 layer, and a Conv_4 layer;

[0021] The Conv_1 layer contains 16 1×1 convolution kernels, the convolution step size is set to 1, and the padding is set to the same;

[0022] The Conv_2 layer contains 16 3×3 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0023] The Conv_3 layer contains 16 5×5 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0024] The Conv_4 layer contains 16 7×7 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0025] The Conv_1 layer, Conv_2 layer, Conv_3 layer, and Conv_4 layer are combined in parallel, that is, the input image will pass through these four layers respectively when passing through the multi-scale convolution module, and then at the end of the module, the output tensors of these four layers are spliced ​​and output in the Concatenate layer;

[0026] Step S23. Construct a multi-scale face prior frame and confidence prediction module, set a series of prior frames of different scales on convolutional feature maps of different scales, these prior frames and multi-scale convolutional layers constitute the last module of the multi-face detection neural network, and the multi-face detection neural network predicts the center offset and length and width offset of each prior frame through this module. Combined with the setting of the prior frame, the predicted value of the face detection frame on the feature map can be obtained.

[0027] As a further improvement, step S23 includes:

[0028] Step S231. Obtain a feature map feature_map_1 of a first scale through the output of the multi-scale convolution module; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the first scale with a step size of 2 to obtain a feature map feature_map_2 of a second scale; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the second scale with a step size of 2 to obtain a feature map feature_map_3 of a third scale;

[0029] Step S232. Based on feature_map_1, three prior boxes are set for each point on it, the scale of the prior box is set to 24, and the aspect ratio is set to {1:2, 1:1, 2:1}, to obtain the prior box information of each point;

[0030] Step S233. Based on feature_map_2, three prior boxes are set for each point on it, the scale of the prior box is set to 32, and the aspect ratio is set to {1:3, 1:1, 3:1}, to obtain the prior box information of each point;

[0031] Step S234. Based on feature_map_3, three prior boxes are set for each point on it, the scale of the prior box is set to 60, and the aspect ratio is set to {1:4, 1:1, 4:1}, to obtain the prior box information of each point;

[0032] Step S235. Based on feature_map_1, the model outputs the category prediction value of each prior box on feature_map_1 through a Conv layer using 6 3×3 convolution kernels with a step size of 1 and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_1 through a Conv layer using 12 3×3 convolution kernels with a step size of 1 and a Reshape layer;

[0033] Step S236. Based on feature_map_2, the model outputs the category prediction value of each prior box on feature_map_2 through a Conv layer using 6 3×3 convolution kernels and a step size of 1 and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_2 through a Conv layer using 12 3×3 convolution kernels and a step size of 1 and a Reshape layer;

[0034] Step S237. Based on feature_map_3, the model outputs the category prediction value of each prior box on feature_map_3 through a Conv layer using 6 3×3 convolution kernels with a step size of 1 and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_3 through a Conv layer using 12 3×3 convolution kernels with a step size of 1 and a Reshape layer;

[0035] Step S238. Aggregate the information of all the prior boxes obtained in steps S232 to S237 as the output of the module. The output tensor shape is (n_boxes, n_classes+8), where n_boxes represents the number of prior boxes, n_classes represents the number of categories, and 8 represents the position parameter value related to the prior box. The neural network trains the model through the output results of this module.

[0036] Further, step S1 includes:

[0037] Step S11. Collecting a set number of face images in natural scenes, including images of single face scenes and multi-face scenes, and face images with complex backgrounds or simple backgrounds;

[0038] Step S12: annotating the face image, using a series of rectangular frames to frame the face image in the image, and recording the position information of the rectangular frames on the face image;

[0039] Step S13. Divide the face images into a training set and a validation set in a ratio of 8:2. The training set is used for learning and training the multi-face detection neural network, and the validation set is used for model evaluation after each round of training iteration to facilitate model parameter adjustment.

[0040] Furthermore, the method also includes step S14 of performing data enhancement processing on each face image in the training set.

[0041] Furthermore, the data enhancement processing includes any one of horizontal flipping or vertical flipping processing, Gaussian blur processing, distorting image local feature processing, and adding Gaussian noise processing.

[0042] Further, step S3 includes:

[0043] Step S31. Converting the real face rectangular frame information marked on the training image of the training set into a format consistent with the prediction result information of the multi-face detection neural network;

[0044] Step S32. Design a loss function. The loss function is divided into two parts. The first part is the model's position prediction error for the prior box, and the second part is the category confidence prediction error. The simplified model of the loss function is shown in the following formula:

[0045]

[0046] Among them, N represents the number of positive samples of the prior frame, L loc Represents the position error, L conf represents the confidence error;

[0047] Step S33. Initialize all parameters of the multi-face detection neural network to Gaussian random numbers with a mean of 0 and a variance of 1;

[0048] Step S34. Set the training iteration number epoch of the multi-face detection neural network, and select the initial learning rate of the multi-face detection neural network according to the iteration number epoch: if epoch>10000, the initial learning rate is set to 10 -4 ; Otherwise, the initial learning rate is set to 10 -3 ; Set the batch_size of each training to 64;

[0049] Step S45. Randomly select batch_size training images from the training set and input them into the multi-face detection neural network, and use the loss function of step S32 to calculate the error between the output value of the multi-face detection neural network and the true value;

[0050] Step S46: Use the gradient descent algorithm to update the weight parameters of the multi-face detection neural network, and increase the number of training iterations by 1;

[0051] Step S47. Determine whether the number of iterations reaches the initially set number of training iterations epochs. If so, the training ends and the weight parameters of the multi-face detection neural network are saved; if not, repeat step S45.

[0052] Further, step S4 includes:

[0053] Step S41. Calculate the parameters of the actual predicted rectangular frame according to the prior frame information and the offset prediction value corresponding to the prior frame;

[0054] Step S42: Convert the position information of the actual predicted rectangular frame into coordinate information;

[0055] Step S43: Mapping the coordinate information of the actual predicted rectangular frame to the original image through a scale transformation to obtain a face prediction candidate frame;

[0056] Step S44. Setting a face confidence threshold. For each face prediction candidate box, if the prediction value of the face prediction candidate box for the face is lower than the face confidence threshold, the face prediction candidate box is removed.

[0057] Step S45. For the face prediction candidate boxes that have passed the screening in step S44, further screening them by non-maximum suppression;

[0058] Step S46. For the face prediction candidate boxes filtered by step S45, set a top_k value, arrange the face prediction candidate boxes according to the size of the prediction value of the face, and obtain the top_k face prediction candidate boxes with the highest scores as the final output of the decoder. These top_k face prediction candidate boxes are the detection results of the real face image.

[0059] Further, step S45 includes:

[0060] Step S451. Setting a candidate box intersection-over-union threshold IOU threshold;

[0061] Step S452. Sort the face prediction scores of each face prediction candidate box from large to small;

[0062] Step S453. The face prediction candidate box with the largest score is taken as the box with the highest confidence, and the remaining face prediction candidate boxes are compared with the box. If the threshold of overlap between a remaining face prediction candidate box and the box is greater than IOUthreshold, it is removed, otherwise it is retained;

[0063] Step S454. For the candidate boxes retained in step S453, repeat step S452 until all boxes are compared.

[0064] In order to achieve the above-mentioned second objective, the present invention provides a computer device, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to implement a multi-face detection method as described above.

[0065] In order to achieve the third objective above, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to implement a multi-face detection method as described above.

[0066] Beneficial Effects

[0067] Compared with the prior art, the present invention has the following advantages:

[0068] The present invention makes a series of improvements based on the convolutional neural network, including the introduction of a dimensionality reduction convolution module and a multi-scale convolution module. Compared with the existing face detection technology, it not only improves the detection speed, but also improves the accuracy of face detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a flow chart of the method of the present invention;

[0070] Figure 2 The present invention is a flowchart of performing multi-face detection. DETAILED DESCRIPTION

[0071] The present invention will be further described below with reference to the specific embodiments in the accompanying drawings.

[0072] See also Figure 1 , 2 , a multi-face detection method, comprising the following steps:

[0073] Step S1. Constructing a face image dataset;

[0074] Step S2. Constructing a multi-face detection neural network;

[0075] Step S3. Iteratively train the multi-face detection neural network according to the face image data set to obtain a multi-face detection model;

[0076] Step S4. Construct a multi-face detection decoder;

[0077] Step S5: input the image to be detected into the multi-face detection model to obtain a prediction result, and input the prediction result into the multi-face detection decoder to obtain a detection result.

[0078] Building a face image dataset includes:

[0079] Step S11. Collect a set number of face images in natural scenes, including images of single face scenes and multi-face scenes, and face images with complex backgrounds or simple backgrounds. The larger the number, the better. For example, the number is set to 100,000.

[0080] Step S12: annotate the face image, use a series of rectangular frames to frame the face image in the image, and record the position information of the rectangular frames on the face image; the position information includes the coordinate information of the center point of the rectangular frame (r x ,r y ), the height of the rectangular frame r h and width r w ;

[0081] Step S13. Divide the face images into a training set and a validation set in a ratio of 8:2. The training set is used for learning and training the multi-face detection neural network, and the validation set is used for model evaluation after each round of training iteration to facilitate model parameter adjustment.

[0082] Step S14: Perform data enhancement processing on each face image in the training set. The data enhancement processing includes any one of horizontal flipping or vertical flipping processing, Gaussian blur processing, distorting the local features of the image, and adding Gaussian noise processing, wherein the sigma value of the Gaussian blur processing is 0 to 3.0, the scale value of the distorted image local features processing is 0.01 to 0.05, and the scale value of the Gaussian noise processing is 0 to 51.

[0083] Building a multi-face detection neural network includes:

[0084] Step S21. Construct a dimensionality reduction convolution module. The dimensionality reduction convolution module is located at the beginning of the network and is used to process the input image of the input layer. The dimensionality reduction convolution module includes a Conv layer, an Activation layer, a BN layer, and a MaxPooling2D layer.

[0085] The Conv layer contains 12 5×5 convolution kernels, the convolution step is set to 4, the padding value is set to the same, and the convolution kernel parameters are initialized using Gaussian random numbers with a mean of 0 and a variance of 1;

[0086] The Activation layer is connected after the Conv layer and uses Relu as the activation function;

[0087] The BN layer is connected after the Activation layer, and the values ​​of gamma_regularizer and beta_regularrizer are set to 0.0001;

[0088] MaxPooling2D is connected after the BN layer, the pooling window size is set to 2×2, the pooling step size is set to 2, and the padding is set to the same; when an input image of size n passes through the convolution layer of the module, it will become 12 feature maps of size n / 4. The size remains unchanged after passing through the Activation layer and the BN layer. After passing through the MaxPooling2D layer, it becomes 12 feature maps of size n / 8. That is, when the input image passes through a dimensionality reduction convolution module, the size will become 1 / 8 of the original size, and the size of the input space has been compressed. Since computers spend more time on convolution operations, this dimensionality reduction operation can effectively improve the computing speed of the network. In addition, setting the convolution window to 5×5 can also reduce the information loss caused by the reduction of spatial size to a certain extent;

[0089] Step S22. Construct a multi-scale convolution module: The multi-scale convolution module includes a Conv_1 layer, a Conv_2 layer, a Conv_3 layer, and a Conv_4 layer;

[0090] The Conv_1 layer contains 16 1×1 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0091] The Conv_2 layer contains 16 3×3 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0092] The Conv_3 layer contains 16 5×5 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0093] The Conv_4 layer contains 16 7×7 convolution kernels, the convolution step is set to 1, and the padding is set to the same;

[0094] The Conv_1 layer, Conv_2 layer, Conv_3 layer, and Conv_4 layer are combined in parallel, that is, the input image will pass through these four layers respectively when passing through the multi-scale convolution module, and then at the end of the module, the output tensors of these four layers are spliced ​​and output in the Concatenate layer; the purpose of this is to efficiently expand the depth and width of the neural network model, improve the accuracy of the model, and prevent the occurrence of overfitting to a certain extent. In addition, this module can also extract image features at different scales and improve the model's ability to aggregate visual information at different scales;

[0095] Step S23. Construct a multi-scale face prior frame and confidence prediction module. Since the face sizes in each face image are different, in order to enable the model to better learn the visual patterns of faces of different scales, a series of prior frames of different scales are set on the convolution feature maps of different scales. These prior frames and multi-scale convolution layers constitute the last module of the multi-face detection neural network. The multi-face detection neural network predicts the center offset and length and width offset of each prior frame through this module. Combined with the setting of the prior frame, the predicted value of the face detection frame on the feature map can be obtained. The specific steps are as follows:

[0096] Step S231. Obtain a feature map feature_map_1 of the first scale through the output of the multi-scale convolution module; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the first scale with a step size of 2 to obtain a feature map feature_map_2 of the second scale; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the second scale with a step size of 2 to obtain a feature map feature_map_3 of the third scale;

[0097] Step S232. Based on feature_map_1, set 3 prior boxes for each point on it, set the scale of the prior box to 24, and set the aspect ratio to {1:2, 1:1, 2:1} to obtain the prior box information of each point; if the size of feature_map_1 is n×n, then n×n×3 prior box information can be obtained, and the prior box information includes the coordinates of the center point of the prior box (b x ,b y ), and the width b of the prior box w and the height b of the prior box h ;

[0098] Step S233. Based on feature_map_2, set 3 prior boxes for each point on it, set the scale of the prior box to 32, and set the aspect ratio to {1:3, 1:1, 3:1} to obtain the prior box information of each point; if the size of feature_map_2 is n×n, then n×n×3 prior box information can be obtained, and the prior box information includes the coordinates of the center point of the prior box (b x ,b y ), and the width b of the prior box w and the height b of the prior box h ;

[0099] Step S234. Based on feature_map_3, set 3 prior boxes for each point on it, set the scale of the prior box to 60, and set the aspect ratio to {1:4, 1:1, 4:1} to obtain the prior box information of each point; if the size of feature_map_3 is n×n, then n×n×3 prior box information can be obtained, and the prior box information includes the coordinates of the center point of the prior box (b x ,b y ), and the width b of the prior box w and the height b of the prior box h ;

[0100] Step S235. Based on feature_map_1, the model outputs the category prediction value of each prior box on feature_map_1 through a Conv layer using 6 3×3 convolution kernels and a step size of 1 and a Reshape layer; then outputs the offset prediction value of each prior box on feature_map_1 through a Conv layer using 12 3×3 convolution kernels and a step size of 1 and a Reshape layer; the offset prediction value includes the center point offset (p x ,p y ) and width offset p w and height offset p h ;

[0101] Step S236. Based on feature_map_2, the model outputs the category prediction value of each prior box on feature_map_2 through a Conv layer using 6 3×3 convolution kernels and a step size of 1 and a Reshape layer; then outputs the offset prediction value of each prior box on feature_map_2 through a Conv layer using 12 3×3 convolution kernels and a step size of 1 and a Reshape layer; the offset prediction value includes the center point offset (p x ,p y ) and width offset p w and height offset p h ;

[0102] Step S237. Based on feature_map_3, the model outputs the category prediction value of each prior box on feature_map_3 through a Conv layer using 6 3×3 convolution kernels and a step size of 1 and a Reshape layer; then outputs the offset prediction value of each prior box on feature_map_3 through a Conv layer using 12 3×3 convolution kernels and a step size of 1 and a Reshape layer; the offset prediction value includes the center point offset (p x ,py ) and width offset p w and height offset p h ;

[0103] Step S238. Aggregate the information of all the prior boxes obtained in steps S232 to S237 as the output of the module. The output tensor shape is (n_boxes, n_classes+8), where n_boxes represents the number of prior boxes, n_classes represents the number of categories, and 8 represents the position parameter value related to the prior box. The neural network trains the model through the output results of this module.

[0104] Iterative training of a multi-face detection neural network includes:

[0105] Step S31. Converting the real face rectangle information marked on the training image of the training set into a format consistent with the prediction result information of the multi-face detection neural network;

[0106] Step S32. Design a loss function. The loss function is divided into two parts. The first part is the model's position prediction error for the prior box, and the second part is the category confidence prediction error. The simplified model of the loss function is shown in the following formula:

[0107]

[0108] Among them, N represents the number of positive samples of the prior frame, L loc Represents the position error, L conf represents the confidence error;

[0109] Step S33. Initialize all parameters of the multi-face detection neural network to Gaussian random numbers with a mean of 0 and a variance of 1;

[0110] Step S34. Set the training iteration number epoch of the multi-face detection neural network, and select the initial learning rate of the multi-face detection neural network according to the iteration number epoch: if epoch>10000, the initial learning rate is set to 10 -4 ; Otherwise, the initial learning rate is set to 10 -3 ; Set the batch_size of each training to 64;

[0111] Step S45. Randomly select batch_size training images from the training set and input them into the multi-face detection neural network. Use the loss function of step S32 to calculate the error between the output value of the multi-face detection neural network and the true value.

[0112] Step S46. Use the gradient descent algorithm to update the weight parameters of the multi-face detection neural network, and increase the number of training iterations by 1;

[0113] Step S47. Determine whether the number of iterations reaches the initially set number of training iterations epochs. If so, the training ends and the weight parameters of the multi-face detection neural network are saved; if not, repeat step S45.

[0114] The output of the neural network model is the predicted value related to the prior frame. In practical applications, these predicted information needs to be mapped back to the original image. In addition, the number of prior frames is usually much larger than the number of faces in the actual image. These prior frames need to be screened and processed. Therefore, a decoder is constructed to perform a series of processing on the output of the model. The specific process is as follows:

[0115] Step S41. Calculate the parameters of the actual predicted rectangular frame according to the prior frame information and the offset prediction value corresponding to the prior frame; specifically, according to the four parameters of the prior frame (b x ,b y ,b w ,b h ) and the four offsets predicted by the network model (p x ,p y ,p w ,p h ) Calculate the parameters of the actual predicted rectangular box as follows:

[0116] t x =b x +p x ×b w

[0117] t y =b y +p y ×b h

[0118]

[0119]

[0120] In the above formula, t x ,t y are the center point coordinates of the actual predicted rectangular box, t w and t h They are the width and height of the actual predicted rectangular box respectively;

[0121] Step S42: Convert the actual predicted rectangular frame position information into coordinate information in the following way:

[0122] t xmin =t x -0.5×t w

[0123] tymin =t y -0.5×t h

[0124] t xmax =t x +0.5×t w

[0125] t ymax =t y +0.5×t h

[0126] In the above formula, t xmin ,t ymin are the horizontal and vertical coordinates of the upper left corner of the actual predicted rectangular box, t xmax and t ymax They are the horizontal and vertical coordinates of the lower right corner of the actual predicted rectangular box respectively;

[0127] Step S43: Map the coordinate information of the actual predicted rectangular frame to the original image through a scale transformation to obtain a face prediction candidate frame, as shown in the following formula:

[0128] x min =t xmin ×i w

[0129] y min =t ymin ×i h

[0130] x max =t xmax ×i w

[0131] y max =t ymax ×i h

[0132] In the formula, (x min ,y min ) is the upper left corner coordinate of the face prediction candidate box on the real image, (x max ,y max ) is the lower right corner coordinate of the face prediction candidate box on the real image, i w is the width of the real image, i his the height of the real image; through the above transformation, the format of the model's output tensor can be converted to (n_boxes, n_classes+4), where n_boxes represents the number of predicted candidate boxes on the real image, the value of n_classes is 2, indicating that the model predicts two probability values ​​of the object in this candidate box as a face or background, and 4 represents the position information of this candidate box; in order to more intuitively display the prediction results of the candidate box, the output format is converted to (n_boxes, class_id+4), where class_id represents the category information, 0 represents the background, and 1 represents the face;

[0133] Step S44. Setting a face confidence threshold. For each face prediction candidate box, if the prediction value of the face prediction candidate box for the face is lower than the face confidence threshold, the face prediction candidate box is removed.

[0134] Step S45. For the face prediction candidate boxes that have passed the screening in step S44, further screening them by non-maximum suppression;

[0135] Step S46. There may still be a large number of face prediction candidate frames screened by step S45. Therefore, a top_k value can be set according to the actual situation of the scene (some scenes have many faces, some scenes have few faces), and the face prediction candidate frames are arranged according to the size of the prediction value of the face. The top_k face prediction candidate frames with the highest scores are obtained as the final output of the decoder. These top_k face prediction candidate frames are the detection results of the real face image.

[0136] Screening them through non-maximum suppression specifically includes:

[0137] Step S451. Setting a candidate box intersection-over-union threshold IOU threshold;

[0138] Step S452. Sort the face prediction scores of each face prediction candidate box from large to small;

[0139] Step S453. The face prediction candidate box with the largest score is taken as the box with the highest confidence. When there are multiple face prediction candidate boxes with the largest scores, the first one is selected, and the remaining face prediction candidate boxes are compared with the box. If the threshold of overlap between a remaining face prediction candidate box and the box is greater than the IOU threshold, it is removed, otherwise it is retained;

[0140] Step S454. For the candidate boxes retained in step S453, repeat step S452 until all boxes are compared.

[0141] The present invention makes a series of improvements based on the convolutional neural network, including the introduction of a dimensionality reduction convolution module and a multi-scale convolution module. Compared with the existing face detection technology, it not only improves the detection speed, but also improves the accuracy of face detection.

[0142] A computer device comprises a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to implement a multi-face detection method as described above.

[0143] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement a multi-face detection method as described above.

[0144] Practical Application

[0145] The process of detecting a face image in the present invention is as follows:

[0146] 1. Adjust the size of the input image to 512×512×3;

[0147] 2. After the input image is input into the multi-face detection model, it first passes through two dimensionality reduction convolution modules to obtain 12 8×8 feature maps. It can be seen that after the dimensionality reduction convolution module, the size of the feature map is reduced by 64 times compared to the size of the original input image;

[0148] 3. The feature map obtained in step 2 is passed through two multi-scale convolution modules to obtain 64 8×8 feature maps. These 64 feature maps are obtained by convolution of convolution kernels of different scales.

[0149] 4. The feature map obtained in step 3 is used as the feature map feature_map_1 of the first scale for face detection; 128 3×3 convolution kernels are used to convolve the feature map of the first scale with a step size of 2 to obtain 128 4×4 feature maps, which are used as the feature map feature_map_2 of the second scale for face detection; 128 3×3 convolution kernels are used to convolve the feature map of the second scale with a step size of 2 to obtain 128 2×2 feature maps, which are used as the feature map feature_map_3 of the third scale for face detection;

[0150] 5. Based on feature_map_1, set 3 prior boxes for each point on it, the scale of the prior box is set to 24, and the aspect ratio is set to {1:2, 1:1, 2:1}, thus obtaining 8×8×3 prior boxes;

[0151] 6. Based on feature_map_2, set 3 prior boxes for each point on it, the scale of the prior box is set to 32, and the aspect ratio is set to {1:3, 1:1, 3:1}, thus obtaining 4×4×3 prior boxes;

[0152] 7. Based on feature_map_3, set 3 prior boxes for each point on it, set the scale of the prior box to 60, and set the aspect ratio to {1:4, 1:1, 4:1}, thus obtaining 2×2×3 prior boxes;

[0153] 8. Based on feature_map_1, a Conv layer (6 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain 8×8×3×2 values, that is, the predicted values ​​of each prior box for the two categories of face and background; another Conv layer (12 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain 8×8×3×4 values, that is, the model's predicted offset values ​​for each prior box, including the center point offset (p x ,p y ) and width offset p w and height offset p h ;

[0154] 9. Based on feature_map_2, a Conv layer (6 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain the category prediction values ​​of 4×4×3×2 prior boxes; another Conv layer (12 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain the offset prediction values ​​of 4×4×3×4 prior boxes, including the center point offset (p x ,p y ) and width offset p w and height offset p h ;

[0155] 10. Based on feature_map_3, a Conv layer (6 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain the category prediction values ​​of 2×2×3×2 prior boxes; another Conv layer (12 3×3 convolution kernels, convolution step size is 1) and a Reshape layer are used to obtain the offset prediction values ​​of 2×2×3×4 prior boxes, including the center point offset (p x ,p y ) and width offset p w and height offset p h ;

[0156] 11. Aggregate the information of all the prior boxes obtained above into the output of the module. The output tensor shape is (252, 2+8), where 252 represents the number of prior boxes, 2 represents the number of categories, and 8 represents the position parameter value related to the prior box;

[0157] 12. Map the prior box position information output by the module to the original input image to obtain an output tensor of (252, 2+4), where 252 represents the number of candidate boxes mapped to the original input image, 2 represents the two probability values ​​of the model predicting the object in this candidate box as a face or background, and 4 represents the position information of the candidate box on the image (x min ,y min ,x max ,y max ), in order to more intuitively display the prediction results of the candidate box, the output format is converted to (252,1+4);

[0158] 13. Set a face confidence threshold and remove candidate boxes whose face prediction scores are lower than the threshold;

[0159] 14. Set a candidate box intersection-over-union ratio threshold IOU threshold, sort the candidate boxes that have passed the screening in step 13 from high to low according to the face prediction score of each candidate box, take the first candidate box as the box with the highest confidence, and compare the remaining candidate boxes with it. If the threshold of overlap between a candidate box and it is greater than the IOU threshold, it will be removed, otherwise it will be retained, and repeat this step until all boxes have been compared;

[0160] Set a top_k value, sort the candidate boxes that have passed the screening in step 14 from large to small according to the face prediction scores of the candidate boxes, and select the top_k candidate boxes as the output results of the model detection face box.

[0161] The above are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A multi-face detection method, characterized in that: The following steps are involved: Step S1. Constructing a face image dataset; Step S2. Constructing a multi-face detection neural network; Step S3. iteratively training the multi-face detection neural network according to the face image data set to obtain a multi-face detection model; Step S4. Construct a multi-face detection decoder; Step S5. Input the image to be detected into the multi-face detection model to obtain a prediction result, and input the prediction result into the multi-face detection decoder to obtain a detection result; Wherein, step S2 comprises: Step S21. Construct a dimensionality reduction convolution module, which is located at the beginning of the network and is used to process the input image of the input layer. The dimensionality reduction convolution module includes a Conv layer, an Activation layer, a BN layer, and a MaxPooling2D layer; The Conv layer contains 12 5×5 convolution kernels, the convolution step is set to 4, the padding value is set to the same, and the convolution kernel parameters are initialized using Gaussian random numbers with a mean of 0 and a variance of 1; The Activation layer is connected after the Conv layer, and Relu is used as the activation function; The BN layer is connected after the Activation layer, and the values ​​of gamma_regularizer and beta_regularrizer are both set to 0.0001; The MaxPooling2D is connected after the BN layer, the pooling window size is set to 2×2, the pooling step size is set to 2, and the padding is set to the same; Step S22. Construct a multi-scale convolution module: the multi-scale convolution module includes a Conv_1 layer, a Conv_2 layer, a Conv_3 layer, and a Conv_4 layer; The Conv_1 layer, Conv_2 layer, Conv_3 layer, and Conv_4 layer are combined in parallel, that is, the input image will pass through these four layers respectively when passing through the multi-scale convolution module, and then at the end of the module, the output tensors of these four layers are spliced ​​and output in the Concatenate layer; Step S23. Construct a multi-scale human face prior frame and confidence prediction module, set a series of prior frames of different scales on convolutional feature maps of different scales, these prior frames and multi-scale convolutional layers constitute the last module of the multi-face detection neural network, and the multi-face detection neural network predicts the center offset and length and width offset of each prior frame through this module, and then combines the setting of the prior frame to obtain the predicted value of the face detection frame on the feature map; Step S23 includes: Step S231. Obtain a feature map feature_map_1 of a first scale through the output of the multi-scale convolution module; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the first scale with a step size of 2 to obtain a feature map feature_map_2 of a second scale; construct a convolution layer, use 128 3×3 convolution kernels and convolve the feature map of the second scale with a step size of 2 to obtain a feature map feature_map_3 of a third scale; Step S232. Based on feature_map_1, set three prior boxes for each point on it to obtain the prior box information of each point; Step S233. Based on feature_map_2, set three prior boxes for each point on it to obtain the prior box information of each point; Step S234. Based on feature_map_3, set three prior boxes for each point on it to obtain the prior box information of each point; Step S235. Based on feature_map_1, the model outputs the category prediction value of each prior box on feature_map_1 through a Conv layer and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_1 through a Conv layer and a Reshape layer; Step S236. Based on feature_map_2, the model outputs the category prediction value of each prior box on feature_map_2 through a Conv layer and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_2 through a Conv layer and a Reshape layer; Step S237. Based on feature_map_3, the model outputs the category prediction value of each prior box on feature_map_3 through a Conv layer and a Reshape layer; and then outputs the offset prediction value of each prior box on feature_map_3 through a Conv layer and a Reshape layer; Step S238. Aggregate the information of all the prior boxes obtained in steps S232 to S237 as the output of the module. The output tensor shape is (n_boxes, n_classes + 8), where n_boxes represents the number of prior boxes, n_classes represents the number of categories, and 8 represents the position parameter value related to the prior box. The neural network trains the model through the output results of this module.

2. A multi-face detection method according to claim 1, characterized in that: Step S1 includes: Step S11. Collecting a set number of face images in natural scenes, including images of single face scenes and multi-face scenes, and face images with complex backgrounds or simple backgrounds; Step S12: annotating the face image, using a series of rectangular frames to frame the face image in the image, and recording the position information of the rectangular frames on the face image; Step S13. Divide the face images into a training set and a validation set in a ratio of 8:

2. The training set is used for learning and training the multi-face detection neural network, and the validation set is used for model evaluation after each round of training iteration to facilitate model parameter adjustment.

3. A multi-face detection method according to claim 2, characterized in that: The method also includes step S14 of performing data enhancement processing on each face image in the training set.

4. A multi-face detection method according to claim 3, characterized in that: The data enhancement processing includes any one of horizontal flipping or vertical flipping processing, Gaussian blur processing, distorting image local feature processing, and adding Gaussian noise processing.

5. The multi-face detection method according to claim 2, characterized in that: Step S3 includes: Step S31. Converting the real face rectangular frame information marked on the training image of the training set into a format consistent with the prediction result information of the multi-face detection neural network; Step S32. Design a loss function. The loss function is divided into two parts. The first part is the model's position prediction error for the prior box, and the second part is the category confidence prediction error. The simplified model of the loss function is shown in the following formula: Among them, N represents the number of positive samples of the prior frame, represents the position error, represents the confidence error; Step S33. Initialize all parameters of the multi-face detection neural network to Gaussian random numbers with a mean of 0 and a variance of 1; Step S34. Set the training iteration number epoch of the multi-face detection neural network, and select the initial learning rate of the multi-face detection neural network according to the iteration number epoch: if epoch>10000, the initial learning rate is set to 10 -4 ; Otherwise, the initial learning rate is set to 10 -3 ; Set the batch_size of each training to 64; Step S45. Randomly select batch_size training images from the training set and input them into the multi-face detection neural network, and use the loss function of step S32 to calculate the error between the output value of the multi-face detection neural network and the true value; Step S46: Use the gradient descent algorithm to update the weight parameters of the multi-face detection neural network, and increase the number of training iterations by 1; Step S47. Determine whether the number of iterations reaches the initially set number of training iterations epochs. If so, the training ends and the weight parameters of the multi-face detection neural network are saved; if not, repeat step S45.

6. A multi-face detection method according to claim 1, characterized in that: Step S4 includes: Step S41. Calculate the parameters of the actual predicted rectangular frame according to the prior frame information and the offset prediction value corresponding to the prior frame; Step S42: Convert the position information of the actual predicted rectangular frame into coordinate information; Step S43: Mapping the coordinate information of the actual predicted rectangular frame to the original image through a scale transformation to obtain a face prediction candidate frame; Step S44. Setting a face confidence threshold. For each face prediction candidate box, if the prediction value of the face prediction candidate box for the face is lower than the face confidence threshold, the face prediction candidate box is removed. Step S45. For the face prediction candidate boxes that have passed the screening in step S44, further screening them by non-maximum suppression; Step S46. For the face prediction candidate boxes filtered by step S45, set a top_k value, arrange the face prediction candidate boxes according to the size of the prediction value of the face, and obtain the top_k face prediction candidate boxes with the highest scores as the final output of the decoder. These top_k face prediction candidate boxes are the detection results of the real face image.

7. A multi-face detection method according to claim 6, characterized in that: Step S45 includes: Step S451. Setting a candidate box intersection-over-union threshold IOU threshold; Step S452. Sort the face prediction scores of each face prediction candidate box from large to small; Step S453. The face prediction candidate box with the largest score is taken as the box with the highest confidence, and the remaining face prediction candidate boxes are compared with the box. If the overlap threshold between a remaining face prediction candidate box and the box is greater than the IOU threshold, it is removed, otherwise it is retained; Step S454. For the candidate boxes retained in step S453, repeat step S452 until all boxes are compared.

8. A computer device, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to implement a multi-face detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement a multi-face detection method as described in any one of claims 1-7.

Images