Face detection method and device based on feature pyramid

Abstract

The invention discloses a face detection method and device based on a feature pyramid. The method comprises the following steps: acquiring a target face image; performing feature extraction on each region of the target face image by using an LP overlapping pooling layer in the face recognition model to obtain first face feature information; performing feature extraction on each region of the target face image by using an adaptive index pooling layer in the face recognition model to obtain second face feature information; fusing the first face feature information and the second face feature information to obtain face feature fusion information; performing feature extraction on the face feature fusion information by using a convolutional layer in a face recognition model to obtain third face feature information; and processing the third face feature information by using a prediction module in the face recognition model to obtain a face detection result corresponding to the target face image. According to the invention, the technical problem of low detection result accuracy of a multi-scale face detection mode used in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for face detection based on a feature pyramid. Background Art

[0002] In view of the multi-scale changes of faces in images, the current mainstream approach in the industry is to use a face detection model based on a feature pyramid. However, the existing feature pyramid-based models still cannot meet the industry's further demand for high accuracy in face detection in complex scenarios such as small-scale face detection and multi-scale multi-face detection. For example, the Cascade CNN model, which uses a cascade structure to construct a face image pyramid, has alleviated the problem of sensitivity to lighting, angles, etc. in open scenes to a certain extent, but it has the upper limit of reducing the number of face detections in high-resolution images, so it cannot be applied to the scene requirements of small-scale multi-faces; the MTCNN model based on a three-layer network structure has a good effect on large-scale single face detection, but because the top-level pyramid associated with the large anchor box in the MTCNN model lacks contextual information and cannot utilize the different potential features of two adjacent scales, when multiple target faces are assigned to different pyramid levels, there will be a problem of missing spatial information, and the global information of each layer of the feature pyramid cannot be utilized, so there is a problem of low accuracy in multi-scale multi-face detection.

[0003] With respect to the problem of low accuracy of detection results of the multi-scale face detection method used in the above-mentioned related technologies, no effective solution has been proposed so far. Summary of the invention

[0004] The embodiment of the present invention provides a face detection method and device based on a feature pyramid, so as to at least solve the technical problem that the multi-scale face detection method used in the related art has a low detection result accuracy.

[0005] According to one aspect of an embodiment of the present invention, a face detection method based on a feature pyramid is provided, comprising: obtaining a target face image, wherein the target face image is an image that needs to be subjected to face detection; performing feature extraction on each region of the target face image using an LP overlapping pooling layer in a face recognition model to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; performing feature extraction on each region of the target face image using an adaptive exponential pooling layer in the face recognition model to obtain second face feature information; fusing the first face feature information and the second face feature information to obtain face feature fusion information; performing feature extraction on the face feature fusion information using a convolutional layer in the face recognition model to obtain third face feature information; and processing the third face feature information using a prediction module in the face recognition model to obtain a face detection result corresponding to the target face image.

[0006] Optionally, the face recognition model uses a ResNet50 network as a backbone network, and the face recognition model uses a convolutional layer that does not change the size of the target face image in each level as a network stage.

[0007] Optionally, the channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

[0008] Optionally, before using the LP overlapping pooling layer in the face recognition model to perform feature extraction on each area of ​​the target face image to obtain first face feature information and using the adaptive exponential pooling layer in the face recognition model to extract feature from each area of ​​the target face image, the feature pyramid-based face detection method also includes: using the convolution block one to perform feature extraction on the target face image to obtain fourth face feature information, and using the convolution block two to perform feature extraction on the target face image to obtain fifth face feature information; splicing the fourth face feature information and the fifth face feature information to obtain spliced ​​face feature information.

[0009] Optionally, the LP overlapping pooling layer in the face recognition model is used to perform feature extraction on each region of the target face image to obtain first face feature information, including: determining that the sliding step size of the LP overlapping pooling layer is 1; performing region segmentation on the target face image based on the sliding step size to obtain multiple overlapping regions of the target face image; and performing feature extraction on the multiple overlapping regions to obtain the first face feature information.

[0010] Optionally, the adaptive index pooling layer in the face recognition model is used to perform feature extraction on each area of ​​the target face image to obtain second face feature information, including: determining the pooling ratio of the DSC pooling layer and the eM pooling in the adaptive index pooling layer; performing the DSC pooling layer and the eM pooling on each area of ​​the target face image according to the pooling ratio to obtain the second face feature information.

[0011] Optionally, the first facial feature information and the second facial feature information are fused to obtain facial feature fusion information, including: processing the first facial feature information using a ReLU activation function to obtain a first weight vector; processing the second facial feature information using a Sigmod activation function to obtain a second weight vector; based on the first weight vector, the second weight vector and the target facial image, fusing the first facial feature information and the second facial feature information to obtain the facial feature fusion information.

[0012] According to another aspect of an embodiment of the present invention, a face detection device based on a feature pyramid is provided, comprising: an acquisition unit, for acquiring a target face image, wherein the target face image is an image that needs to be subjected to face detection; a first extraction unit, for performing feature extraction on each region of the target face image using an LP overlapping pooling layer in a face recognition model to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, wherein each set of training data in the multiple sets of training data comprises: a sample face image and a sample of the sample face image detection result; a second extraction unit, used to use the adaptive exponential pooling layer in the face recognition model to perform feature extraction on each area of ​​the target face image to obtain second face feature information; a fusion unit, used to fuse the first face feature information and the second face feature information to obtain face feature fusion information; a third extraction unit, used to use the convolution layer in the face recognition model to perform feature extraction on the face feature fusion information to obtain third face feature information; a processing unit, used to use the prediction module in the face recognition model to process the third face feature information to obtain the face detection result corresponding to the target face image.

[0013] Optionally, the face recognition model uses a ResNet50 network as a backbone network, and the face recognition model uses a convolutional layer that does not change the size of the target face image in each level as a network stage.

[0014] Optionally, the channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

[0015] Optionally, the face detection device based on feature pyramid also includes: a fourth extraction unit, used to perform feature extraction on each area of ​​the target face image using the LP overlapping pooling layer in the face recognition model to obtain first face feature information and before performing feature extraction on each area of ​​the target face image using the adaptive exponential pooling layer in the face recognition model, perform feature extraction on the target face image using the convolution block one to obtain fourth face feature information, and perform feature extraction on the target face image using the convolution block two to obtain fifth face feature information; a splicing unit, used to splice the fourth face feature information and the fifth face feature information to obtain spliced ​​face feature information.

[0016] Optionally, the first extraction unit includes: a first determination module, used to determine that the sliding step size of the LP overlapping pooling layer is 1; a segmentation module, used to perform region segmentation on the target face image based on the sliding step size to obtain multiple overlapping regions of the target face image; and an extraction module, used to perform feature extraction on the multiple overlapping regions to obtain the first face feature information.

[0017] Optionally, the second extraction unit includes: a second determination module, used to determine the pooling ratio of the DSC pooling layer and the eM pooling in the adaptive exponential pooling layer; a pooling module, used to perform DSC pooling layer and eM pooling on each area of ​​the target face image according to the pooling ratio to obtain the second face feature information.

[0018] Optionally, the fusion unit includes: a first processing module, used to process the first facial feature information using a ReLU activation function to obtain a first weight vector; a second processing module, used to process the second facial feature information using a Sigmod activation function to obtain a second weight vector; and a fusion module, used to fuse the first facial feature information and the second facial feature information based on the first weight vector, the second weight vector and the target facial image to obtain the facial feature fusion information.

[0019] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the above-mentioned face detection methods based on feature pyramid.

[0020] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer instructions, which, when executed by a processor, perform any one of the above-mentioned face detection methods based on feature pyramids.

[0021] In an embodiment of the present invention, a target face image is obtained, wherein the target face image is an image that needs to be subjected to face detection; a LP overlapping pooling layer in a face recognition model is used to perform feature extraction on each region of the target face image to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; a feature extraction is performed on each region of the target face image using an adaptive exponential pooling layer in the face recognition model to obtain second face feature information; the first face feature information and the second face feature information are fused to obtain face feature fusion information; a convolutional layer in the face recognition model is used to perform feature extraction on the face feature fusion information to obtain third face feature information; and a prediction module in the face recognition model is used to process the third face feature information to obtain a face detection result corresponding to the target face image. Through the technical solution provided by the present invention, it is achieved by constructing a multi-scale face detection model, using the new LP overlapping pooling and adaptive exponential pooling to fully utilize the global information of the feature map while paying attention to important features, and finally by introducing channel attention to concentrate the important features of the image into the weight vector, and retain the valuable features by multiplying with the original feature map; the adaptive pooling spatial attention module fully mines the channel information through adaptive pooling, and uses the spatial attention mechanism to integrate contextual information for the purpose of face detection, thereby improving the accuracy of face detection, and thus solving the technical problem of low accuracy of detection results in the multi-scale face detection method used in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 It is a hardware structure block diagram of a mobile terminal of a face detection method based on a feature pyramid according to an embodiment of the present invention;

[0024] Figure 2 is a flow chart of a face detection method based on a feature pyramid according to an embodiment of the present invention;

[0025] Figure 3 is an overall structural block diagram of a face detection method of a feature pyramid according to an embodiment of the present invention;

[0026] Figure 4 is a structural diagram of a channel enhancement module based on channel attention according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of LP overlapping pooling according to an embodiment of the present invention;

[0028] Figure 6 is a structural diagram of an adaptive pooling spatial attention module according to an embodiment of the present invention;

[0029] Figure 7 is a structural diagram of a context module according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of a face detection device based on a feature pyramid according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Face detection is an important technology in the field of computer vision and is the basic step for tasks such as face recognition, facial image enhancement and face analysis. The realization of practical applications such as security monitoring, identity verification and human-computer interaction all rely on face detection technology.

[0034] The essence of face detection is to detect the positions corresponding to all faces in an image based on a target detection model. At present, face detection is mainly based on a target detection model of a deep convolutional neural network, which can learn powerful deep features and automatically process feature selection, and has significant performance improvements. Many existing face detection networks, such as the SSD model and the Fast-RCNN model, inherit the network design used for general target (non-face) detection.

[0035] However, compared with general object detection, face detection must cope with more extreme scale variations of faces in images. The resolution of faces may vary from about 10 pixels to thousands of pixels, and how to detect a large number of small-scale faces in an image and multiple faces with large scale differences in a single image has become a major challenge.

[0036] As described above, the multi-scale face detection method used in the related art has a low detection accuracy. In view of the above defects, a face detection method and device based on a feature pyramid, a computer-readable storage medium, and a computer program product are provided in the embodiments of the present invention.

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0038] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a face detection method based on a feature pyramid according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0039] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the face detection method based on feature pyramid in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal through a network. Examples of the above network include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks and combinations thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0040] According to an embodiment of the present invention, a method embodiment of a face detection method based on a feature pyramid is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] Figure 2 is a flow chart of a face detection method based on a feature pyramid according to an embodiment of the present invention. Figure 2 As shown, the face detection method based on feature pyramid includes the following steps:

[0042] Step S202, obtaining a target face image, wherein the target face image is an image that needs to be subjected to face detection.

[0043] In this embodiment, the face image of the object that needs face detection can be collected by an image acquisition device. For example, the face image of the object moving to the security door can be collected by a camera set at the security door. Of course, it can also be an image obtained from other terminal devices with image acquisition functions. In the embodiment of the present invention, the method of obtaining the face image is not specifically limited, and it can be an image obtained in various forms.

[0044] Step S204, using the LP overlapping pooling layer in the face recognition model to extract features from each area of ​​the target face image to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image.

[0045] Figure 3 is an overall structural block diagram of a face detection method according to a feature pyramid of an embodiment of the present invention. Figure 3 As shown in the figure, it adopts a new feature enhancement pyramid network, which effectively integrates the features of high and low layers, and fully utilizes spatial information in multi-scale target detection scenarios by introducing multi-scale convolution.

[0046] For example, important features can be highlighted by introducing LP overlapping pooling. Among them, LP overlapping pooling is a pooling operation used in deep learning, which has higher efficiency and better performance than traditional pooling operations. The main feature of LP overlapping pooling is that when performing a pooling operation, the step size of the pooling window is smaller than the size of the pooling window, so that the pooling operation can produce overlapping parts, thereby improving the efficiency of feature extraction. Through LP overlapping pooling, the size of the feature map can be reduced while retaining important feature information, thereby improving the robustness and generalization ability of the network. In addition, LP overlapping pooling can also reduce the risk of overfitting and improve the training speed and convergence speed of the network.

[0047] Among them, feature pyramid is a method for multi-scale image analysis and feature extraction. It decomposes the image into sub-images of different scales, extracts features at each scale, and finally combines these features into a pyramid-shaped structure. Its advantage is that it can capture details and structural information of different scales in the image, thereby improving the accuracy and robustness of image analysis and recognition.

[0048] In this embodiment, the LP overlapping pooling layer in the face recognition model can be used to extract features from each region of the target face image to obtain first face feature information. The first face feature information here not only reduces the size of the feature map, but also retains important feature information.

[0049] Step S206, using the adaptive exponential pooling layer in the face recognition model to extract features from each region of the target face image to obtain second face feature information.

[0050] In this embodiment, adaptive index pooling is also introduced to extract features from each region of the target face image to avoid information loss.

[0051] like Figure 3As shown, in an embodiment of the present invention, a channel attention module is added to fully utilize the channel relationship between features, and a context module is introduced to each layer of the feature pyramid to increase the receptive field, so as to more effectively learn dense space transformation, thereby improving the detection accuracy in small-scale target scenes.

[0052] Another example Figure 3 As shown in the figure, the structure of the multi-scale face detection model based on feature pyramid can be divided into 6 parts, namely: bottom-up process, channel enhancement module (lateral connection process), adaptive pooling spatial attention module, top-down process, context module, and multi-task learning module. Among them, the channel enhancement module, adaptive pooling spatial attention module, context module, and multi-task learning module are the core modules of the model. The specific contents of each part of the overall structure of the multi-scale face detection model based on feature pyramid are as follows:

[0053] (1) Bottom-up process:

[0054] The model uses the ResNet50 network as the backbone network.

[0055] The bottom-up process is a common feedforward propagation process of convolutional neural networks. Each level uses downsampling with a scaling step of 2 from the bottom up to obtain a feature layer composed of feature maps of different scales. The convolutional layers that do not change the scale of the feature map in each level are classified as one network stage. Figure 3 As shown, a total of four network stages are adopted, namely C2, C3, C4, and C5.

[0056] (2) Channel enhancement module:

[0057] In order to better adapt to the small target face detection scenario and extract spatial features more effectively, a channel enhancement module based on multi-scale convolution, LP overlapping convolution, and adaptive exponential pooling is used to improve the lateral connection part of FPN, such as Figure 4 As shown, Figure 4 is a structural diagram of a channel enhancement module based on channel attention according to an embodiment of the present invention.

[0058] In the existing feature pyramid model, the lateral connection is usually designed as a 1*1 convolution layer. The reduction in the number of feature channels in each layer will lead to the loss of some spatial information. At the same time, the 1*1 convolution layer also has the problem of low receptive field.

[0059] Based on the above defects, in an embodiment of the present invention, the face recognition model adopts the ResNet50 network as the backbone network, and the face recognition model uses the convolution layer that does not change the size of the target face image in each level as a network stage.

[0060] In addition, in an embodiment of the present invention, the channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

[0061] That is, in order to increase the receptive field and thus improve the detection effect of small target faces, a set of 1*1 and 3*3 convolution blocks are used in the channel enhancement module, where different convolution kernel sizes provide different receptive fields, such as Figure 2 As shown in the figure. The reason why two 3*3 convolutions are used instead of 5*5 convolutions is to avoid introducing too many parameters due to the increase of the convolution kernel, which will lead to overfitting of the model. After passing through this set of 1*1 and 3*3 convolution blocks, the obtained features are spliced ​​together. In this process, each layer of features has the same number of channels to ensure the fusion of subsequent features.

[0062] Step S208: Fusing the first facial feature information and the second facial feature information to obtain facial feature fusion information.

[0063] In this embodiment, the first facial feature information and the second facial feature information may be spliced ​​together to obtain spliced ​​facial feature fusion information, thereby obtaining relatively comprehensive facial feature information.

[0064] Step S210, using the convolutional layer in the face recognition model to perform feature extraction on the face feature fusion information to obtain third face feature information.

[0065] like Figure 4 As shown in Figure 2, after concatenating the facial feature information, a new channel attention module is introduced to correct the features and retain the valuable features. Figure 4 As shown, firstly, two pooling methods, LP overlapping pooling and adaptive exponential pooling, are used to compress spatial information.

[0066] Step S212, using the prediction module in the face recognition model to process the third face feature information to obtain a face detection result corresponding to the target face image.

[0067] As can be seen from the above, in an embodiment of the present invention, after acquiring a target face image, the LP overlapping pooling layer in the face recognition model is used to perform feature extraction on each region of the target face image to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network, and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; the adaptive exponential pooling layer in the face recognition model is used to perform feature extraction on each region of the target face image to obtain second face feature information; the first face feature information and the second face feature information are fused to obtain face feature fusion information; the convolution in the face recognition model is used The first layer performs feature extraction on the facial feature fusion information to obtain the third facial feature information; the prediction module in the face recognition model is used to process the third facial feature information to obtain the face detection result corresponding to the target face image, thereby achieving a multi-scale face detection model, using the new LP overlapping pooling and adaptive exponential pooling to fully utilize the global information of the feature map while paying attention to important features, and finally focusing the important features of the image into the weight vector by introducing channel attention, and retaining the valuable features by multiplying with the original feature map; the adaptive pooling spatial attention module fully mines the channel information through adaptive pooling, and uses the spatial attention mechanism to integrate contextual information for the purpose of face detection, thereby improving the accuracy of face detection.

[0068] It is worth noting that in the embodiment of the present invention, the multi-scale face detection technology based on the feature pyramid belongs to the field of target detection. Based on the feature pyramid as the main architecture, a newly designed channel enhancement module, an adaptive pooling spatial attention module and a context module are introduced to design and implement a multi-scale face detection model. The channel enhancement model is applied to the lateral connection of the feature pyramid, and the spatial information loss problem caused by the 1×1 convolution is overcome by multi-scale convolution. A new LP overlapping pooling and adaptive exponential pooling are introduced to fully utilize the global information of the feature map while paying attention to important features. Finally, the important features of the image are concentrated in the weight vector by introducing channel attention, and valuable features are retained by multiplying with the original feature map; the adaptive pooling spatial attention module fully mines the channel information through adaptive pooling, and integrates the context information by using the spatial attention mechanism, thereby making up for the defect that the top-level features fail to utilize the feature information of other layers; the context module is applied to the output feature map of each stage of the top-down process of the feature pyramid to increase the receptive field and more effectively learn the dense spatial network conversion, thereby improving the ability of small target face detection.

[0069] Therefore, the technical solution provided by the embodiment of the present invention solves the technical problem of low accuracy of detection results in the multi-scale face detection method used in the related art.

[0070] According to the above embodiment of the present invention, before using the LP overlapping pooling layer in the face recognition model to perform feature extraction on each area of ​​the target face image to obtain first face feature information and using the adaptive exponential pooling layer in the face recognition model to extract features from each area of ​​the target face image, the feature pyramid-based face detection method also includes: using convolution block one to perform feature extraction on the target face image to obtain fourth face feature information, and using convolution block two to perform feature extraction on the target face image to obtain fifth face feature information; splicing the fourth face feature information and the fifth face feature information to obtain spliced ​​face feature information.

[0071] In this embodiment, the 1*1 convolution block 1 and the 3*3 convolution block 2 can be used to extract features of the target face image to obtain different face feature information, and these face feature information are spliced. After splicing, a new channel attention module is introduced to correct the features and retain the valuable features. Figure 4 As shown, firstly, two pooling methods, LP overlapping pooling and adaptive exponential pooling, are used to compress spatial information.

[0072] According to the above-mentioned embodiment of the present invention, the LP overlapping pooling layer in the face recognition model is used to perform feature extraction on each region of the target face image to obtain first face feature information, including: determining the sliding step size of the LP overlapping pooling layer to be 1; performing region segmentation on the target face image based on the sliding step size to obtain multiple overlapping regions of the target face image; and performing feature extraction on the multiple overlapping regions to obtain first face feature information.

[0073] It should be noted that LP overlapping pooling is a new pooling method proposed in the embodiment of the present invention. It combines the advantages of LP pooling and overlapping pooling. It can not only enable each area in the feature map to participate in multiple feature extractions to obtain a more comprehensive feature representation, but also obtain a flexible compromise between maximum pooling and average pooling, which can not only focus on the strongest features, but also achieve the effect of smoothing and averaging features.

[0074] The difference between LP overlapping pooling and standard pooling is that the sliding step size is set to 1, which is smaller than the size of the pooling kernel. Then each area in the feature map will participate in multiple feature extraction tasks, such as Figure 5 As shown ( Figure 5 is a schematic diagram of LP overlapping pooling according to an embodiment of the present invention). For a 4×4 feature map, a 3×3 matrix is ​​obtained after an LP overlapping pooling with a pooling kernel of 2×2. For a pooling area, LP overlapping pooling calculates the p-norm of the feature in the pooling window, as shown in formula (1). When p=2, an LP overlapping pooling is as follows Figure 5shown. Where y represents the pooled value obtained after the LP overlapping pooling operation. n: represents the number of elements in the feature map covered by the pooling kernel. In the example mentioned in the text, the pooling kernel is 2×2, so n=4, represents the sum of the p-th power of the absolute value of xi, To normalize the parameters, the above steps can be used to perform pooling operations on specific pooling areas.

[0075] According to the above-mentioned embodiment of the present invention, the adaptive index pooling layer in the face recognition model is used to perform feature extraction on each area of ​​the target face image to obtain second face feature information, including: determining the pooling ratio of the DSC pooling layer and the eM pooling in the adaptive index pooling layer; performing the DSC pooling layer and the eM pooling on each area of ​​the target face image according to the pooling ratio to obtain the second face feature information.

[0076] In the embodiment of the present invention, adaptive index pooling comprehensively utilizes the characteristics of DSC pooling and eM pooling, which can not only avoid the interference of outliers in average pooling on pooling, but also highlight activation values ​​with greater influence without losing important information. The calculation formula of adaptive index pooling is shown in formula (2), where β controls the ratio of DSC pooling and eM pooling. The calculation formulas of DSC pooling and eM pooling are shown in formulas (3)-(5). DSC pooling assumes that geometrically close observations are more similar than geometrically farther observations. By using each a i To the pooling window mean Similarity (as shown in formula (4)), and the similarity index between the activation vector and the average activation is used to calculate the weighted kernel value, as shown in formula (3). The calculation of eM pooling is shown in formula (5), which is generated by summing all weighted activations in the kernel area. This pooling method can highlight activations with greater influence without the risk of losing important information.

[0077]

[0078]

[0079]

[0080]

[0081] The adaptive exponential pooling formula (Formula 2) is the core of adaptive exponential pooling, which combines the results of DSC pooling and eM pooling respectively through weights. This combination allows the model to flexibly choose a more suitable pooling strategy in different situations. The DSC pooling formula (Formula 3) weights the sum by calculating the similarity weight of each activation relative to the mean of the pooling window. This weighting mechanism enables the model to assign different weights according to the similarity of the activation value to the mean, thereby better capturing the local structural information in the feature map. The DSC pooling similarity weight formula (Formula 4) quantifies the geometric similarity of the activation values ​​by considering the modulus and dot product of the activation values. The eM pooling formula weights the sum by calculating the weight of each activation. The weight is determined by the exponential function of the activation value, which helps the model highlight those activations with larger values, which may correspond to important features in the feature map. The above formula model can more accurately capture the important features of the input data. The similarity weight of DSC pooling can reduce the impact of outliers on the pooling results and improve robustness. eM pooling helps to highlight those activation values ​​that have a greater impact on the final task, which can improve the model's sensitivity to key information. Although the adaptive exponential pooling calculation is slightly complicated, it provides a way to balance performance and computational cost, which can improve the performance of the model without significantly increasing the computational burden. By taking advantage of the two pooling mechanisms, the generalization ability of the model on unseen data can be improved. In the forward propagation process of the neural network, a more robust and representative feature representation is generated to provide support for subsequent classification or other tasks.

[0082] According to the above embodiment of the present invention, the first facial feature information and the second facial feature information are fused to obtain facial feature fusion information, including: using the ReLU activation function to process the first facial feature information to obtain a first weight vector; using the Sigmod activation function to process the second facial feature information to obtain a second weight vector; based on the first weight vector, the second weight vector and the target facial image, the first facial feature information and the second facial feature information are fused to obtain facial feature fusion information.

[0083] In this embodiment, after performing the above-mentioned LP overlapping pooling and adaptive index pooling, the global information of the feature map is obtained, information loss is prevented, and important features are highlighted. The features obtained after pooling by the two pooling methods are respectively generated into two weight vectors by ReLU and Sigmod functions. Subsequently, the two weight vectors after LP overlapping pooling and adaptive index pooling are multiplied with the original feature map, so that the important features of the image are concentrated together. Finally, the feature map is combined with the input feature map to output the feature map.

[0084] In addition, if Figure 6 ( Figure 6), in order to make up for the defect that the top-level features of the existing feature pyramid fail to utilize the feature information of other layers, a residual branch is added to the top-level C5 of the bottom-up path of the feature pyramid, and an adaptive pooling spatial attention module is introduced to extract feature maps at different scales to provide more contextual information, thereby assisting the interaction between the top-level features and the features of other layers.

[0085] In order to integrate rich contextual information, an adaptive pooling pyramid is first constructed. The adaptive pooling pyramid can obtain multiple contextual features at different scales, including global information and sub-region information. The model can fuse information from different receptive fields of different sub-regions.

[0086] The adaptive pooling pyramid module integrates features at multiple pyramid scales. The topmost pyramid layer uses global pooling with a 1*1 pooling kernel, and the remaining pyramid levels from top to bottom use pooling kernels that cover all, half, and a small part of the entire image, dividing the feature map into different sub-regions and forming pooling representations for different positions, outputting feature maps of increasing sizes from top to bottom. Then, the low-dimensional feature map is directly upsampled by bilinear interpolation to obtain features of the same size as the original feature map, and these feature maps are spliced ​​in the channel dimension, as shown in formula (6). In the formula, Kernel i Represents pooling kernels of different sizes, and Adapool represents adaptive pooling operation.

[0087] In the formula, Concat represents the concatenation operation on the channel dimension, which concatenates feature maps of different scales to fuse multi-scale information. Upsample represents the upsampling operation, which is used to interpolate and enlarge the low-resolution feature map to the same size as the original feature map. AdaPool represents the adaptive pooling operation, which is performed according to the feature map C5 and the pooling kernels Kerneli of different scales. C5 represents the feature map input to the adaptive pooling pyramid module, which is usually the output of a layer in the deep learning model. Kerneli represents the pooling kernels of different scales, which are used to perform pooling operations in pyramid layers of different levels. Through the upsampling Upsample operation, feature maps of different scales are adjusted to the same size as the original feature map for subsequent processing. The Concat operation is used to concatenate feature maps of different scales on the channel dimension to achieve feature fusion, which increases the amount of information that the network can use. AdaPool allows the model to dynamically adjust the pooling behavior according to the content of the feature map C5, which increases the flexibility of the model. Through this multi-scale and adaptive pooling strategy, the model can generate richer and more robust feature representations, which helps improve the performance of downstream tasks. The design of the adaptive pooling pyramid module allows the model to process features of different scales in a computationally efficient manner, avoiding the need to process full-size feature maps for each scale separately.

[0088] After concatenating the feature maps, a spatial attention module is further introduced so that each feature map f output from the adaptive pooling pyramid i The corresponding spatial weights are generated by sequentially passing through the 1*1 convolution layer, ReLU activation layer, 3*3 convolution layer and Sigmoid activation layer. Each spatial weight represents the features of the feature maps of different scales obtained by the previous pooling. The Hadamard product operation is performed through different spatial weights and the concatenated feature maps. The final output is the top layer P6 in the top-down process of the feature pyramid, as shown in formula (7). In the formula, Conv(f1) represents the feature map f i A simplified representation of the spatial weights obtained after a convolution operation.

[0089] P6 is endowed with multi-scale contextual information, which enhances the interaction between the top-level features and the features of the rest of the pyramid layers.

[0090]

[0091] Conv(·) represents a convolution operation, which is used to learn spatial weights from feature maps. fi represents the i-th feature map output by the adaptive pooling pyramid. ⊙ represents the Hadamard product, which is the element-wise product. ⊕ represents element-wise addition, which is used to combine the weighted sums of different feature maps. f represents the concatenated feature map, which is obtained by concatenating multiple fi. n represents the number of feature maps output by the adaptive pooling pyramid. Through the spatial attention module, each fifi feature map generates spatial weights through convolution and activation layers, which can highlight the most important parts of the feature map. The generated spatial weights are combined with the original feature map f through the Hadamard product, allowing P6 to focus on the key areas in the feature map. Since fi comes from pooling operations of different scales, P6 integrates multi-scale information and enhances the expressiveness of features. In this way, P6 enhances the interaction between the top-level features and the features of the remaining layers in the pyramid, which helps to improve the model's understanding and integration of features of different scales.

[0092] It should be noted that in the embodiment of the present invention, the top-down process enables the semantic information of high-level features to be transferred to the underlying features, so that the shallow feature map contains contextual information of different scales. The i-th layer in the top-down process upsamples the feature map, and the output features obtained after the channel enhancement module of the i-1th layer in the bottom-up process are added element by element, and the upsampled map is merged with the corresponding bottom-up map. Iterate the above process, and obtain P5, P4, P3, and P2 from top to bottom respectively.

[0093] In addition, in the embodiment of the present invention, in order to improve the accuracy of small target face detection, the present invention increases the receptive field of the corresponding layer and learns dense spatial transformation more effectively by modeling the context. Therefore, after the top-down process outputs {P2, P3, P4, P5, P6}, the context module is introduced, such as Figure 7 ( Figure 7 : is a structural diagram of the context module according to an embodiment of the present invention), where W represents the width of the feature map, that is, the number of pixels in the horizontal direction; H represents the height of the feature map, that is, the parameter in the vertical direction; C represents the number of channels of the feature map, that is, the number at each pixel position. Drawing on the idea of ​​introducing contextual information by expanding the window around the candidate proposal in the two-stage detector, larger convolution kernels can be used, such as 5*5 and 7*7 convolution kernels. However, the 5*5 and 7*7 convolution kernels will increase the amount of parameters and increase the amount of calculation. In order to reduce the amount of parameters, a series of 3*3 convolution kernels are used in the context module to replace larger convolution kernels, such as Figure 5 shown.

[0094] For P iThe output features of the image are first passed through a 3*3 convolution to reduce the number of channels to half of the original number. Then the output after the maximum pooling is divided into two paths, which are respectively sent to one 3*3 convolution and two 3*3 convolutions. The features after the two convolution operations are concatenated for the final prediction task. Modeling the context in this way will increase the receptive field proportionally with the step size of the corresponding layer, thereby increasing the target scale of each detection module, thereby improving the accuracy of small-scale face detection and multi-scale face detection.

[0095] In the embodiment of the present invention, after the context module, the feature map is used for the final prediction task after passing through the 1x1 convolution layer. A multi-task loss is used in the prediction task. The multi-task loss includes tasks such as face classification, bounding box regression, face five-point landmark regression and center regression to train the model, as shown in formula (8): Where, L cls Represents the classification loss. The classification loss adopts focal loss and can be divided into two categories: face foreground and background, where p i Represents the predicted probability of an anchor box, is the actual category label of the anchor box. box represents the box positioning loss, using CIoU loss, t i , Represent the predicted and actual bounding boxes respectively. L pts represents the landmark localization loss, using smooth L1 loss, where l i , are the predicted and actual facial landmark points respectively. cen Represents the centrality loss, using binary cross entropy loss, where v i , are the predicted center position and the actual center position, respectively. Considering the different contribution levels of different training anchor boxes to the box positioning loss, reflecting the difference in their distances from the corresponding object center, the model adds dynamic weights according to their centrality. In the inference phase, this centrality loss branch will be removed and no additional computational cost will be introduced. The parameters λ1 and λ2 in formula (8) are used to balance these loss terms.

[0096] It should be noted that the technical solution provided by the embodiment of the present invention can be applied to visual tasks based on face analysis, such as face recognition, person-document comparison, human-computer interaction, and facial emotion analysis. It is particularly suitable for face detection scenarios with multiple multi-scale and small-scale targets in an image. Through the newly designed channel enhancement module, adaptive pooling spatial attention module and context module, the existing face detection model is improved in the insensitivity of small-scale and multi-scale target detection and the lack of spatial information, thereby achieving high-accuracy multi-scale face detection in complex environments.

[0097] From the above description, it can be seen that the technical solution provided by the embodiment of the present invention designs, implements and optimizes the model for multi-scale face detection to meet the high-precision requirements of face detection and recognition in practical applications. In order to achieve this goal, this method designs and implements a multi-scale face detection model with high accuracy.

[0098] Here, the face detection model based on feature enhancement pyramid uses feature pyramid as the basic skeleton, introduces a new channel enhancement module in the lateral connection of feature pyramid to optimize the feature representation of each network stage of feature pyramid, and makes full use of global spatial information in multi-scale target detection scenario through multi-scale convolution, avoiding the problem of spatial information loss in existing face detection methods. A new LP overlapping pooling and adaptive exponential pooling method are introduced to avoid losing key information in feature maps while highlighting important features, and further retain valuable features by introducing channel attention; by introducing adaptive pooling spatial attention module, multi-scale contextual information is given to top-level features, enhancing the interaction between top-level features and feature pyramid; by introducing context module, the receptive field is increased and dense spatial network transformation is learned more effectively in small face detection scenarios, and multi-scale context learning provides more effective multi-scale facial features for the model, thereby improving the problem that existing face detection models are insensitive to small-scale targets, and finally achieving high-accuracy face detection effect.

[0099] Moreover, the technical solution provided by the embodiment of the present invention can make full use of multi-scale features and global spatial information in the scenario where there are multiple small-scale and multi-scale targets in the image, and can effectively improve the defect of low multi-scale detection accuracy of the existing face detection model. It has strong multi-feature learning ability and robustness, and can meet the high accuracy requirements of detecting faces of different scales in practical applications such as face recognition and analysis.

[0100] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0102] According to an embodiment of the present invention, a face detection device based on a feature pyramid for implementing the above-mentioned face detection method based on a feature pyramid is also provided. Figure 8 is a schematic diagram of a face detection device based on a feature pyramid according to an embodiment of the present invention. Figure 8 As shown, the face detection device based on feature pyramid includes: an acquisition unit 801, a first extraction unit 803, a second extraction unit 805, a fusion unit 807, a third extraction unit 809 and a processing unit 811.

[0103] The acquisition unit 801 is used to acquire a target face image, wherein the target face image is an image that needs to be subjected to face detection.

[0104] The first extraction unit 803 is used to use the LP overlapping pooling layer in the face recognition model to extract features from each area of ​​the target face image to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image.

[0105] The second extraction unit 805 is used to extract features from each region of the target face image using the adaptive exponential pooling layer in the face recognition model to obtain second face feature information.

[0106] The fusion unit 807 is used to fuse the first facial feature information and the second facial feature information to obtain facial feature fusion information.

[0107] The third extraction unit 809 is used to extract features from the face feature fusion information using the convolution layer in the face recognition model to obtain third face feature information.

[0108] The processing unit 811 is used to process the third face feature information using the prediction module in the face recognition model to obtain a face detection result corresponding to the target face image.

[0109] It should be noted here that the above-mentioned acquisition unit 801, first extraction unit 803, second extraction unit 805, fusion unit 807, third extraction unit 809 and processing unit 811 correspond to steps S202 to S212 in the above-mentioned embodiments, and the five units have the same instances and application scenarios implemented by the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0110] As can be seen from the above, in the scheme recorded in the above-mentioned embodiment of the present invention, an acquisition unit can be used to acquire a target face image, wherein the target face image is an image that needs to be subjected to face detection; a first extraction unit is used to perform feature extraction on each region of the target face image using the LP overlapping pooling layer in the face recognition model to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; a second extraction unit is used to perform feature extraction on each region of the target face image using the adaptive exponential pooling layer in the face recognition model to obtain second face feature information; a fusion unit is used to fuse the first face feature information and the second face feature information to obtain face feature fusion information; using the third extraction unit to extract the face feature fusion information using the convolution layer in the face recognition model to obtain the third face feature information; and using the processing unit to process the third face feature information using the prediction module in the face recognition model to obtain the face detection result corresponding to the target face image, thereby achieving the goal of building a multi-scale face detection model, using the new LP overlapping pooling and adaptive exponential pooling to fully utilize the global information of the feature map while paying attention to important features, and finally focusing the important features of the image into the weight vector by introducing channel attention, and retaining the valuable features by multiplying the original feature map; the adaptive pooling spatial attention module fully mines the channel information through adaptive pooling, and uses the spatial attention mechanism to integrate contextual information for the purpose of face detection, thereby improving the accuracy of face detection.

[0111] Therefore, the technical solution provided by the embodiment of the present invention solves the technical problem of low accuracy of detection results in the multi-scale face detection method used in the related art.

[0112] In an optional embodiment, the face recognition model uses a ResNet50 network as a backbone network, and the face recognition model uses a convolutional layer that does not change the size of the target face image in each level as a network stage.

[0113] In an optional embodiment, the channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

[0114] In an optional embodiment, the feature pyramid-based face detection device also includes: a fourth extraction unit, used to perform feature extraction on the target face image using convolution block one to obtain fourth face feature information before using the LP overlapping pooling layer in the face recognition model to extract features from each region of the target face image to obtain first face feature information and using the adaptive exponential pooling layer in the face recognition model to extract features from each region of the target face image to obtain fourth face feature information, and use convolution block two to extract features from the target face image to obtain fifth face feature information; a splicing unit, used to splice the fourth face feature information and the fifth face feature information to obtain spliced ​​face feature information.

[0115] In an optional embodiment, the first extraction unit includes: a first determination module, used to determine that the sliding step size of the LP overlapping pooling layer is 1; a segmentation module, used to perform region segmentation on the target face image based on the sliding step size to obtain multiple overlapping regions of the target face image; and an extraction module, used to perform feature extraction on multiple overlapping regions to obtain first face feature information.

[0116] In an optional embodiment, the second extraction unit includes: a second determination module, used to determine the pooling ratio of the DSC pooling layer and the eM pooling in the adaptive exponential pooling layer; a pooling module, used to perform DSC pooling layer and eM pooling on each area of ​​the target face image according to the pooling ratio to obtain second face feature information.

[0117] In an optional embodiment, the fusion unit includes: a first processing module, used to process the first facial feature information using a ReLU activation function to obtain a first weight vector; a second processing module, used to process the second facial feature information using a Sigmod activation function to obtain a second weight vector; and a fusion module, used to fuse the first facial feature information and the second facial feature information based on the first weight vector, the second weight vector and the target facial image to obtain facial feature fusion information.

[0118] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes any one of the above-mentioned face detection methods based on feature pyramids.

[0119] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0120] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining a target face image, wherein the target face image is an image that requires face detection; using the LP overlapping pooling layer in the face recognition model to extract features from each area of ​​the target face image to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; using the adaptive exponential pooling layer in the face recognition model to extract features from each area of ​​the target face image to obtain second face feature information; fusing the first face feature information and the second face feature information to obtain face feature fusion information; using the convolutional layer in the face recognition model to extract features from the face feature fusion information to obtain third face feature information; using the prediction module in the face recognition model to process the third face feature information to obtain a face detection result corresponding to the target face image.

[0121] Optionally, the face recognition model adopts a ResNet50 network as a backbone network, and the face recognition model uses a convolutional layer that does not change the size of the target face image in each level as a network stage.

[0122] Optionally, the channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

[0123] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: before using the LP overlapping pooling layer in the face recognition model to perform feature extraction on each region of the target face image to obtain first face feature information and using the adaptive exponential pooling layer in the face recognition model to perform feature extraction on each region of the target face image, using convolution block one to perform feature extraction on the target face image to obtain fourth face feature information, and using convolution block two to perform feature extraction on the target face image to obtain fifth face feature information; the fourth face feature information and the fifth face feature information are spliced ​​to obtain spliced ​​face feature information.

[0124] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: determining that the sliding step size of the LP overlapping pooling layer is 1; performing region segmentation on the target face image based on the sliding step size to obtain multiple overlapping regions of the target face image; performing feature extraction on the multiple overlapping regions to obtain first face feature information.

[0125] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: determining a pooling ratio of the DSC pooling layer and the eM pooling in the adaptive exponential pooling layer; performing DSC pooling layer and eM pooling on each area of ​​the target face image according to the pooling ratio to obtain second face feature information.

[0126] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: processing the first facial feature information using a ReLU activation function to obtain a first weight vector; processing the second facial feature information using a Sigmod activation function to obtain a second weight vector; and fusing the first facial feature information and the second facial feature information based on the first weight vector, the second weight vector, and the target facial image to obtain facial feature fusion information.

[0127] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer instructions, which, when executed by a processor, execute any one of the above-mentioned face detection methods based on feature pyramids.

[0128] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0129] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0134] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A face detection method based on feature pyramid, characterized in that: include: Acquire a target face image, wherein the target face image is an image that needs to be subjected to face detection; Using the LP overlapping pooling layer in the face recognition model to extract features from each area of ​​the target face image, obtaining first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; Using the adaptive exponential pooling layer in the face recognition model to extract features from each region of the target face image to obtain second face feature information; Fusing the first facial feature information and the second facial feature information to obtain facial feature fusion information; Using the convolutional layer in the face recognition model to perform feature extraction on the face feature fusion information to obtain third face feature information; The prediction module in the face recognition model is used to process the third face feature information to obtain a face detection result corresponding to the target face image.

2. The face detection method based on feature pyramid according to claim 1, characterized in that: The face recognition model uses a ResNet50 network as a backbone network, and the face recognition model uses a convolutional layer that does not change the size of the target face image in each level as a network stage.

3. The face detection method based on feature pyramid according to claim 2, characterized in that: The channel enhancement module of the face recognition model includes a convolution block 1 with a size of 1*1 and a convolution block 2 with a size of 3*3.

4. The face detection method based on feature pyramid according to claim 3, characterized in that: Before extracting features from each region of the target face image using the LP overlapping pooling layer in the face recognition model to obtain first face feature information and extracting features from each region of the target face image using the adaptive exponential pooling layer in the face recognition model, the method further includes: Using the convolution block 1 to perform feature extraction on the target face image to obtain fourth face feature information, and using the convolution block 2 to perform feature extraction on the target face image to obtain fifth face feature information; The fourth facial feature information and the fifth facial feature information are spliced ​​together to obtain spliced ​​facial feature information.

5. The face detection method based on feature pyramid according to claim 1, characterized in that: The LP overlapping pooling layer in the face recognition model is used to extract features from each region of the target face image to obtain first face feature information, including: Determine that the sliding step size of the LP overlapping pooling layer is 1; Performing region segmentation on the target facial image based on the sliding step length to obtain a plurality of overlapping regions of the target facial image; Feature extraction is performed on the multiple overlapping areas to obtain the first facial feature information.

6. The face detection method based on feature pyramid according to claim 1, characterized in that: Using the adaptive exponential pooling layer in the face recognition model to extract features from each region of the target face image to obtain second face feature information, including: Determining a pooling ratio of the DSC pooling layer and the eM pooling in the adaptive exponential pooling layer; Perform DSC pooling layer and eM pooling on each region of the target facial image according to the pooling ratio to obtain the second facial feature information.

7. The face detection method based on feature pyramid according to claim 1, characterized in that: The first facial feature information and the second facial feature information are fused to obtain facial feature fusion information, including: Processing the first facial feature information using a ReLU activation function to obtain a first weight vector; Processing the second facial feature information using a Sigmoid activation function to obtain a second weight vector; Based on the first weight vector, the second weight vector and the target facial image, the first facial feature information and the second facial feature information are fused to obtain the facial feature fusion information.

8. A face detection device based on feature pyramid, characterized in that: include: An acquisition unit, used for acquiring a target face image, wherein the target face image is an image that needs to be subjected to face detection; A first extraction unit is used to extract features from each region of the target face image using an LP overlapping pooling layer in a face recognition model to obtain first face feature information, wherein the face recognition model is a model based on a feature pyramid network and is obtained through machine learning training using multiple sets of training data, and each set of training data in the multiple sets of training data includes: a sample face image and a sample detection result of the sample face image; A second extraction unit is used to extract features from each region of the target face image using an adaptive exponential pooling layer in the face recognition model to obtain second face feature information; a fusion unit, configured to fuse the first facial feature information and the second facial feature information to obtain facial feature fusion information; A third extraction unit, configured to perform feature extraction on the face feature fusion information by using a convolutional layer in the face recognition model to obtain third face feature information; A processing unit is used to process the third facial feature information using the prediction module in the facial recognition model to obtain a face detection result corresponding to the target facial image.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the face detection method based on feature pyramid as described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by the processor, the face detection method based on the feature pyramid described in any one of claims 1 to 7 is performed.

Images