Method and device for determining face detection model for edge device

Abstract

The invention discloses a face detection model determination method and device for edge equipment, and the method comprises the steps: inputting a face image into a to-be-trained face detection model for training, and obtaining a loss parameter and a model weight of the trained model; when the loss parameter of the model after this training does not meet the condition, updating the model weight of the previous model and continuing training until the loss parameter meets the condition; and executing a scene test based on the to-be-tested model meeting the loss parameter condition to obtain a model accuracy rate, and when the model accuracy rate meets an accuracy rate condition, determining that the to-be-tested model is a trained model. Therefore, the method can improve the determination accuracy of the face detection model, does not need to use a full connection layer and repeatedly perform pooling operation on the model in the training process, reduces the model training parameters, guarantees the model precision, improves the model training efficiency, obtains a lightweight model, and improves the user experience. Therefore, the balance between the model identification precision and the network scale is achieved.

Description

Technical Field

[0001] The present invention relates to the field of face detection technology, and in particular to a method and device for determining a face detection model for an edge device. Background Art

[0002] As the most promising and widely used biometric authentication method, face detection algorithms have received widespread attention in the field of computer vision in recent years. The goal of face detection is to find the corresponding positions of all faces in an image or video and output the coordinates of the face's circumscribed rectangle in the image or video. With the development of deep learning technology, face detection algorithms have made great progress. For face detection on edge devices, existing methods are usually implemented by deploying pre-trained deep learning models on edge devices. At present, the main face detection algorithms used for edge devices in the field of deep learning are MTCNN, RetinaFace, and ULFGFD algorithms.

[0003] However, due to the equipment conditions of edge devices, face detection models deployed on edge devices often find it difficult to achieve a balance between accuracy and network scale. On the one hand, in order to maintain a high amount of model parameters, the face detection model of edge devices is difficult to reduce to less than 1MB, which causes the face detection model to exceed the processing capacity of the edge device during recognition, resulting in a slower recognition speed and even inability to process in real time; on the other hand, if the amount of parameters is reduced through model compression technology, the recognition accuracy will be significantly reduced, which cannot meet the needs of actual face recognition applications. Therefore, it is particularly important to provide a method for determining a face detection model for edge devices to improve the balance between the accuracy of the face detection model for edge devices and the network scale. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for determining a face detection model for an edge device, which can improve the balance between the accuracy of the face detection model for the edge device and the network scale.

[0005] In order to solve the above technical problem, the first aspect of the present invention discloses a method for determining a face detection model for an edge device, the method comprising:

[0006] Inputting the target face images in the target face image data set into the face detection model to be trained, and obtaining the loss parameters and model weights of the face detection model after the training; the target face image data set includes the target face images for model training of the face detection model; the face detection model to be trained has a corresponding initial model weight;

[0007] When it is determined that the loss parameter of the face detection model after the current training does not satisfy the loss parameter condition matched therewith, the model weight of the face detection model after the previous training is updated based on the loss parameter of the face detection model after the current training to obtain the model weight of the face detection model after the current training;

[0008] The face detection model after this training is trained based on the target face images in the target face image data set that are not involved in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training meets the loss parameter condition matched therewith;

[0009] Performing a scene test based on the model weight corresponding to the face detection model to be tested that meets the loss parameter condition matched therewith, to obtain the model accuracy of the face detection model to be tested;

[0010] When it is determined that the model accuracy meets the model accuracy condition matched therewith, it is determined that the face detection model to be tested is the trained face detection model.

[0011] As an optional implementation, in the first aspect of the present invention, the target face image in the target face image data set is input into the face detection model to be trained for training, and the loss parameter and model weight of the face detection model after the training are obtained, including:

[0012] Inputting the target face image in the target face image data set into the face detection model to be trained for training, and obtaining a final feature map of the target face image in the target face image data set;

[0013] Based on the final feature map of each target face image, determining all predicted face bounding boxes of the target face image;

[0014] Based on all the real face bounding boxes annotated in each of the target face images and all the predicted face bounding boxes of the target face image, the loss parameters and model weights of the face detection model after this training are obtained.

[0015] As an optional implementation, in the first aspect of the present invention, before inputting the target face image in the target face image data set into the face detection model to be trained for training and obtaining the loss parameter and model weight of the face detection model after the training, the method further includes:

[0016] Determine at least one target initial face image from the initial face image data set; for each of the target initial face images, determine a plurality of source images corresponding to the target initial face image; the plurality of source images at least include the target initial face image or an augmented face image of the target initial face image after data augmentation;

[0017] For each target initial face image, the corresponding multiple source images are spliced ​​in a minimum connection manner to obtain a combined image corresponding to the target initial face image; the minimum connection manner means that when splicing the multiple source images, the nearest edges between any two adjacent source images completely overlap or partially overlap, or only one vertex between any two adjacent source images overlaps;

[0018] Setting a cropping frame corresponding to the target initial face image according to a preset size;

[0019] For each of the target initial face images, selecting a target point on its corresponding combined image;

[0020] For each of the target initial face images, based on the corresponding cropping frame, the corresponding combined image is cropped, and based on the target point, the cropped combined image is completed to obtain an intermediate face image corresponding to the target initial face image;

[0021] For all the first face images in the first face image data set, some of the first face images are selected to perform image transformation operations to obtain a target face image data set; the first face image data set includes all the target initial face images and all the intermediate face images.

[0022] As an optional implementation, in the first aspect of the present invention, the inputting the target face image in the target face image data set into the face detection model to be trained for training to obtain the final feature map of the target face image in the target face image data set includes:

[0023] Input the target face image in the target face image data set into a face detection model including a plurality of feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; when the feature extraction unit is a deep feature extraction unit, the feature map corresponding to the feature extraction unit is a deep feature map; when the feature extraction unit is a shallow feature extraction unit, the feature map corresponding to the feature extraction unit is a shallow feature map;

[0024] For any of the target facial images, feature fusion is performed on the feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image; the feature map corresponding to the target facial image includes the shallow feature map and the deep feature map.

[0025] As an optional implementation manner, in the first aspect of the present invention, for any of the target facial images, feature fusion is performed on the feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image, including:

[0026] For any of the target facial images, the deep feature map corresponding to the target facial image extracted by the feature extraction unit is input into the SPPF layer for processing to obtain a processed deep feature map corresponding to the target facial image;

[0027] For any of the target facial images, feature fusion is performed on the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image.

[0028] As an optional implementation, in the first aspect of the present invention, for any of the target facial images, feature fusion is performed on the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image, including:

[0029] For any of the target facial images, among the multiple feature maps corresponding to the target facial image, any feature map whose size is larger than that of the deep feature map and smaller than that of the shallow feature map is determined as a middle-layer feature map corresponding to the target facial image;

[0030] For any of the target facial images, splicing the processed deep feature map corresponding to the target facial image with the middle feature map corresponding to the target facial image to obtain a first spliced ​​feature map corresponding to the target facial image;

[0031] Splicing the first splicing feature map corresponding to the target face image with the shallow feature map corresponding to the target face image to obtain a second splicing feature map corresponding to the target face image;

[0032] Adding the first splicing feature map corresponding to the target facial image and the second splicing feature map corresponding to the target facial image element by element to obtain a first added feature map corresponding to the target facial image;

[0033] The first added feature map corresponding to the target facial image and the processed deep feature map corresponding to the target facial image are added element by element to obtain a final feature map corresponding to the target facial image.

[0034] As an optional implementation, in the first aspect of the present invention, the loss parameter of the face detection model after the current training is determined based on at least one of the positioning loss parameter of the face detection model after the current training, the target confidence loss parameter of the face detection model after the current training, and the classification loss parameter of the face detection model after the current training;

[0035] The calculation method of the positioning loss parameter of the face detection model after this training is as follows:

[0036]

[0037] Among them, L CIoU (b,b gt ) represents the positioning loss parameter of the face detection model after this training, b represents the predicted face bounding box corresponding to the target face image predicted by the face detection model after this training, and b gt represents the real face bounding box annotated by the target face image, CIOU represents the complete intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, IOU represents the intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, d u represents the distance between the center point of the predicted face bounding box predicted by the face detection model after this training and the center point of the real face bounding box, d c represents the maximum value of the diagonal distance between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, v represents the aspect ratio coefficient between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, arctan represents the inverse tangent function, and w gt represents the width of the real face bounding box, h gt represents the height of the real face bounding box, b w represents the width of the predicted face bounding box predicted by the face detection model after this training, b h Represents the height of the predicted face bounding box predicted by the face detection model after this training;

[0038] The target confidence loss parameter of the face detection model after this training is calculated as follows:

[0039]

[0040] Among them, L obj (po ,p iou ) represents the target confidence loss parameter of the face detection model after this training; p p represents the probability that the predicted face bounding box contains a face predicted by the face detection model after this training, p iou represents the IOU confidence of the predicted face bounding box predicted by the face detection model after this training, w obj Represents the representation parameter corresponding to the target confidence loss parameter of the face detection model after this training; represents that when calculating the target confidence loss parameter of the face detection model after this training based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1];

[0041] The classification loss parameters of the face detection model after this training are calculated as follows:

[0042]

[0043] Among them, L cls (c p ,c gt ) represents the classification loss parameter of the face detection model after this training; c p represents the probability that the category predicted by the face detection model after this training is accurate, c gt represents the real category label of the target face image, w cls Represents the representation parameter corresponding to the classification loss parameter of the face detection model after this training; It represents that when calculating the classification loss parameter of the face detection model after the current training based on the binary cross entropy, the output probability is limited to the interval [0,1] using the Sigmoid activation function.

[0044] A second aspect of the present invention discloses a device for determining a face detection model for an edge device, the device comprising:

[0045] The first training module is used to input the target face image in the target face image data set into the face detection model to be trained, and obtain the loss parameter and model weight of the face detection model after the training; the target face image data set includes the target face image for model training of the face detection model; the face detection model to be trained has a corresponding initial model weight;

[0046] An updating module, configured to update the model weights of the face detection model after the previous training based on the loss parameters of the face detection model after the current training, when it is determined that the loss parameters of the face detection model after the current training do not meet the loss parameter conditions matched therewith, to obtain the model weights of the face detection model after the current training;

[0047] A second training module is used to train the face detection model after this training based on the target face images in the target face image data set that have not participated in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training meets the loss parameter condition matched therewith;

[0048] A testing module, used to perform a scene test based on a model weight corresponding to the face detection model to be tested that satisfies the loss parameter condition matched therewith, to obtain a model accuracy rate of the face detection model to be tested;

[0049] The first determination module is used to determine that the face detection model to be tested is the trained face detection model when it is determined that the model accuracy meets the model accuracy condition matched therewith.

[0050] As an optional implementation, in the second aspect of the present invention, the first training module inputs the target face image in the target face image data set into the face detection model to be trained for training, and the specific method of obtaining the loss parameter and model weight of the face detection model after this training includes:

[0051] Inputting the target face image in the target face image data set into the face detection model to be trained for training, and obtaining a final feature map of the target face image in the target face image data set;

[0052] Based on the final feature map of each target face image, determining all predicted face bounding boxes of the target face image;

[0053] Based on all the real face bounding boxes annotated in each of the target face images and all the predicted face bounding boxes of the target face image, the loss parameters and model weights of the face detection model after this training are obtained.

[0054] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0055] A second determination module is used to determine at least one target initial face image from the initial face image data set before the first training module inputs the target face image in the target face image data set into the face detection model to be trained for training and obtains the loss parameter and model weight of the face detection model after this training; for each of the target initial face images, determine a plurality of source images corresponding to the target initial face image; the plurality of source images at least include the target initial face image or an augmented face image of the target initial face image after data augmentation;

[0056] a splicing module, for each target initial face image, splicing the corresponding multiple source images in a minimum connection manner to obtain a combined image corresponding to the target initial face image; the minimum connection manner means that when splicing the multiple source images, the nearest edges between any two adjacent source images completely overlap or partially overlap, or only one vertex between any two adjacent source images overlaps;

[0057] A setting module, used to set a cropping frame corresponding to the target initial face image according to a preset size;

[0058] A selection module, for selecting a target point on the corresponding combined image for each target initial face image;

[0059] A cropping module, for cropping the corresponding combined image of each target initial face image based on the corresponding cropping frame, and completing the cropped combined image based on the target point to obtain an intermediate face image corresponding to the target initial face image;

[0060] The transformation module is used to select some of the first face images from all the first face images in the first face image data set and perform image transformation operations to obtain a target face image data set; the first face image data set includes all the target initial face images and all the intermediate face images.

[0061] As an optional implementation, in the second aspect of the present invention, the first training module inputs the target face image in the target face image data set into the face detection model to be trained for training, and the specific method of obtaining the final feature map of the target face image in the target face image data set includes:

[0062] Input the target face image in the target face image data set into a face detection model including a plurality of feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; when the feature extraction unit is a deep feature extraction unit, the feature map corresponding to the feature extraction unit is a deep feature map; when the feature extraction unit is a shallow feature extraction unit, the feature map corresponding to the feature extraction unit is a shallow feature map;

[0063] For any of the target facial images, feature fusion is performed on the feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image; the feature map corresponding to the target facial image includes the shallow feature map and the deep feature map.

[0064] As an optional implementation, in the second aspect of the present invention, the first training module performs feature fusion on the feature map corresponding to the target face image for any of the target face images, and obtains a final feature map corresponding to the target face image in a specific manner including:

[0065] For any of the target facial images, the deep feature map corresponding to the target facial image extracted by the feature extraction unit is input into the SPPF layer for processing to obtain a processed deep feature map corresponding to the target facial image;

[0066] For any of the target facial images, feature fusion is performed on the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image.

[0067] As an optional implementation, in the second aspect of the present invention, the first training module performs feature fusion on the processed deep feature map corresponding to the target face image and the shallow feature map corresponding to the target face image for any of the target face images, and obtains the final feature map corresponding to the target face image in a specific manner including:

[0068] For any of the target facial images, among the multiple feature maps corresponding to the target facial image, any feature map whose size is larger than that of the deep feature map and smaller than that of the shallow feature map is determined as a middle-layer feature map corresponding to the target facial image;

[0069] For any of the target facial images, splicing the processed deep feature map corresponding to the target facial image with the middle feature map corresponding to the target facial image to obtain a first spliced ​​feature map corresponding to the target facial image;

[0070] Splicing the first splicing feature map corresponding to the target face image with the shallow feature map corresponding to the target face image to obtain a second splicing feature map corresponding to the target face image;

[0071] Adding the first splicing feature map corresponding to the target facial image and the second splicing feature map corresponding to the target facial image element by element to obtain a first added feature map corresponding to the target facial image;

[0072] The first added feature map corresponding to the target facial image and the processed deep feature map corresponding to the target facial image are added element by element to obtain a final feature map corresponding to the target facial image.

[0073] As an optional implementation, in the second aspect of the present invention, the loss parameter of the face detection model after the current training is determined based on at least one of the positioning loss parameter of the face detection model after the current training, the target confidence loss parameter of the face detection model after the current training, and the classification loss parameter of the face detection model after the current training;

[0074] The calculation method of the positioning loss parameter of the face detection model after this training is as follows:

[0075]

[0076] Among them, L CIoU (b,b gt ) represents the positioning loss parameter of the face detection model after this training, b represents the predicted face bounding box corresponding to the target face image predicted by the face detection model after this training, and b gt represents the real face bounding box annotated by the target face image, CIOU represents the complete intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, IOU represents the intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, d u represents the distance between the center point of the predicted face bounding box predicted by the face detection model after this training and the center point of the real face bounding box, d c represents the maximum value of the diagonal distance between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, v represents the aspect ratio coefficient between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, arctan represents the inverse tangent function, and w gt represents the width of the real face bounding box, h gt represents the height of the real face bounding box, bw represents the width of the predicted face bounding box predicted by the face detection model after this training, b h Represents the height of the predicted face bounding box predicted by the face detection model after this training;

[0077] The target confidence loss parameter of the face detection model after this training is calculated as follows:

[0078]

[0079] Among them, L obj (p o ,p iou ) represents the target confidence loss parameter of the face detection model after this training; p o represents the probability that the predicted face bounding box contains a face predicted by the face detection model after this training, p iou represents the IOU confidence of the predicted face bounding box predicted by the face detection model after this training, w obj Represents the representation parameter corresponding to the target confidence loss parameter of the face detection model after this training; represents that when calculating the target confidence loss parameter of the face detection model after this training based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1];

[0080] The classification loss parameters of the face detection model after this training are calculated as follows:

[0081]

[0082] Among them, L cls (c p ,c gt ) represents the classification loss parameter of the face detection model after this training; c p represents the probability that the category predicted by the face detection model after this training is accurate, c gt represents the real category label of the target face image, w cls Represents the representation parameter corresponding to the classification loss parameter of the face detection model after this training; It represents that when calculating the classification loss parameter of the face detection model after the current training based on the binary cross entropy, the output probability is limited to the interval [0,1] using the Sigmoid activation function.

[0083] A third aspect of the present invention discloses another device for determining a face detection model for an edge device, the device comprising:

[0084] A memory storing executable program code;

[0085] a processor coupled to the memory;

[0086] The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for determining a face detection model for an edge device disclosed in the first aspect of the present invention.

[0087] The fourth aspect of the present invention discloses a computer-storable medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the method for determining a face detection model for an edge device disclosed in the first aspect of the present invention.

[0088] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0089] The present invention inputs the target face image into the face detection model to be trained for training, and obtains the loss parameter and model weight of the model after this training; when the loss parameter of the model after this training does not meet the loss parameter condition, the model weight of the model after the previous training is updated; the model after this training is trained based on the target face image that does not participate in the model training and the model weight of the model after this training, until the loss parameter meets the condition; based on the satisfied model to be tested, the scene test is performed to obtain the model accuracy; when it is judged that the model accuracy meets the condition, the model to be tested is determined to be a trained face detection model. It can be seen that the present invention can improve the accuracy of the face detection model, which is conducive to improving the accuracy of face recognition; and in the process of training the face detection model, there is no need to use a fully connected layer and repeatedly perform pooling operations on the model, which reduces the training parameters of the model, while ensuring the recognition accuracy of the model, improving the efficiency of model training and obtaining a lightweight model, thereby achieving a balance between model recognition accuracy and network scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0091] Figure 1 It is a flowchart of a method for determining a face detection model for an edge device disclosed in an embodiment of the present invention;

[0092] Figure 2It is a flowchart of another method for determining a face detection model for an edge device disclosed in an embodiment of the present invention;

[0093] Figure 3 It is a structural schematic diagram of a device for determining a face detection model for an edge device disclosed in an embodiment of the present invention;

[0094] Figure 4 It is a structural schematic diagram of another device for determining a face detection model for an edge device disclosed in an embodiment of the present invention;

[0095] Figure 5 It is a structural schematic diagram of another device for determining a face detection model for an edge device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0096] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0098] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0099] The present invention discloses a method and device for determining a face detection model for edge devices, the method comprising: inputting a target face image into a face detection model to be trained for training, and obtaining the loss parameter and model weight of the model after this training; when the loss parameter of the model after this training does not meet the condition, updating the model weight of the previous model and continuing the training until the loss parameter meets the loss parameter condition; performing a scene test based on the model to be tested that meets the loss parameter condition to obtain the model accuracy; when the model accuracy meets the accuracy condition, determining that the model to be tested is a trained model. It can be seen that the present invention can determine the final model based on the model accuracy, and reduces the use of fully connected layers and repeated pooling operations, thereby ensuring the model's recognition accuracy for faces on the premise of reducing model parameters, and achieving a balance between model recognition accuracy and network scale. The following are detailed descriptions.

[0100] Embodiment 1

[0101] See also Figure 1 , Figure 1 1 is a flow chart of a method for determining a face detection model for an edge device disclosed in an embodiment of the present invention. Figure 1 The method shown can be applied to application scenarios for training and determining face detection models for edge devices, such as model construction for specific face recognition, security systems, etc., which is not limited in the embodiments of the present invention. Figure 1 As shown, the method for determining the face detection model for the edge device may include the following operations:

[0102] 101. Input the target face image in the target face image data set into the face detection model to be trained to obtain the loss parameter and model weight of the face detection model after the training;

[0103] In an embodiment of the present invention, a target face image dataset includes target face images for training a face detection model; the face detection model to be trained has corresponding initial model weights; the face detection model to be trained can be a constructed awFace network model, which is a deep learning-based network structure designed by the present invention for face detection tasks on edge devices.

[0104] 102. When it is determined that the loss parameter of the face detection model after the current training does not satisfy the loss parameter condition matched therewith, the model weight of the face detection model after the previous training is updated based on the loss parameter of the face detection model after the current training to obtain the model weight of the face detection model after the current training;

[0105] 103. The face detection model after this training is trained based on the target face images in the target face image dataset that are not involved in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training satisfies the loss parameter condition matching it;

[0106] In the embodiment of the present invention, the loss parameter refers to the parameter value of the loss function, and the loss parameter condition refers to the parameter value of the loss function converging to the requirement and the convergence speed is normal. If the loss parameter still cannot meet the matching loss parameter condition, the network structure and parameters can be changed and retrained.

[0107] 104. Perform a scene test based on the model weight corresponding to the face detection model to be tested that meets the loss parameter condition matched therewith, and obtain the model accuracy of the face detection model to be tested;

[0108] In an embodiment of the present invention, performing a scene test may be performing real-time face data collection, where the collected content includes pictures and videos, the collection scenes are indoor corridors and doorways, and performing a performance test of the model in an actual scene.

[0109] 105. When it is determined that the model accuracy meets the matching model accuracy condition, the face detection model to be tested is determined to be a trained face detection model.

[0110] In the embodiment of the present invention, the model accuracy condition refers to that the model accuracy reaches a preset accuracy threshold. If the accuracy does not meet the model accuracy condition, the network structure and parameters can be changed and retrained.

[0111] In the embodiment of the present invention, the number of network parameters after training is about 250K, which is small enough not to cause excessive burden on edge device deployment. In the test, the face detection task can be completed relatively well in outdoor, corridor and indoor scenes. It should be noted that when the loss parameter of the face detection model after this training meets the matching loss parameter condition, the model weight corresponding to the face detection model to be tested that meets the matching loss parameter condition is executed in step 104 to perform the scene test.

[0112] It can be seen that the present invention can input the target face image into the face detection model to be trained for training, and obtain the loss parameter and model weight of the model after this training; when the loss parameter of the model after this training does not meet the loss parameter condition, update the model weight of the model after the previous training; train the model after this training based on the target face image that did not participate in the model training and the model weight of the model after this training until the loss parameter meets the condition; perform scene testing based on the satisfied model to be tested to obtain the model accuracy; when it is judged that the model accuracy meets the condition, determine that the model to be tested is a trained face detection model. It can be seen that the present invention can improve the accuracy of the face detection model, which is conducive to improving the accuracy of face recognition; and in the process of training the face detection model, there is no need to use a fully connected layer and repeatedly perform pooling operations on the model, which reduces the training parameters of the model, while ensuring the recognition accuracy of the model, improving the efficiency of model training and obtaining a lightweight model, thereby achieving a balance between model recognition accuracy and network scale.

[0113] In an optional embodiment, the step 101 of inputting the target face image in the target face image data set into the face detection model to be trained to obtain the loss parameter and model weight of the face detection model after the training may include:

[0114] Inputting the target face image in the target face image data set into the face detection model to be trained for training, and obtaining a final feature map of the target face image in the target face image data set;

[0115] Based on the final feature map of each target face image, determine all predicted face bounding boxes of the target face image;

[0116] Based on all real face bounding boxes annotated for each target face image and all predicted face bounding boxes for the target face image, the loss parameters and model weights of the trained face detection model are obtained.

[0117] In this optional embodiment, determining all predicted face bounding boxes of the target face image means extracting features and sending them to the detector for processing to obtain multiple target candidate boxes. Further, the determination of the predicted face bounding box can be processed by the non-maximum suppression method on the target candidate box; first, all candidate boxes are sorted by confidence, and the candidate box with the highest confidence is selected; then all remaining candidate boxes are traversed, and the IoU overlap area of ​​the current highest confidence candidate box is compared with the set threshold, and the judgment is made according to the following results:

[0118] a. When the overlapping area of ​​the highest confidence candidate box is greater than the threshold, the current candidate box is retained.

[0119] b. When the overlapping area of ​​the highest confidence candidate box is less than the threshold, the current candidate box is deleted.

[0120] Repeat the above steps and process all candidate boxes in turn to obtain the best target candidate box and determine it as the predicted face bounding box.

[0121] It can be seen that this optional embodiment can input the target face image into the face detection model to be trained for training, obtain the final feature map of the target face image in the target face image data set, and determine the predicted face bounding box based on the final feature map. The loss parameters and model weights of the face detection model after this training are obtained in combination with the real face bounding box, which is beneficial to improving the training accuracy of the face detection model, and further ensuring the model's recognition accuracy of the face while reducing the model parameters.

[0122] In another optional embodiment, in the above step, the target face image in the target face image data set is input into the face detection model to be trained for training, and the final feature map of the target face image in the target face image data set is obtained, which may include:

[0123] Input the target face image in the target face image data set into a face detection model including multiple feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; when the feature extraction unit is a deep feature extraction unit, the feature map corresponding to the feature extraction unit is a deep feature map; when the feature extraction unit is a shallow feature extraction unit, the feature map corresponding to the feature extraction unit is a shallow feature map;

[0124] For any target face image, the feature map corresponding to the target face image is feature fused to obtain a final feature map corresponding to the target face image; the feature map corresponding to the target face image includes a shallow feature map and a deep feature map.

[0125] In this optional embodiment, the feature extraction unit is used to extract sufficient feature information so that the model can learn the feature representation of the face area; the number of feature extraction units can be modified accordingly according to the size of the input image. The deeper the network (the more feature extraction units), the greater the computational cost, so a comprehensive decision is required.

[0126] In this optional embodiment, before the target facial image is input into a face detection model comprising multiple feature extraction units for feature extraction, the target facial image can first be sent to a scale scaling unit for preprocessing. The function of the scale scaling unit is to scale the output scale to 1 / 4 of the original scale, which can reduce the amount of model calculation and speed up the processing speed.

[0127] In this optional embodiment, the activation function of the awFace network model can be set during the training process, and the SiLU activation function is added in the process of extracting the feature map by each feature extraction unit. The SiLU activation function performs a nonlinear transformation on each feature point in the feature map. The feature map processed by the activation function has a stronger nonlinear expression ability and can more effectively capture complex data patterns and features. The expression of the SiLU activation function is:

[0128] SiLU(x)=x×Sigmoid(x)

[0129] Among them, the output of the previous layer is used as the independent variable x, Sigmoid() represents the Sigmoid activation function; the SiLU activation function can introduce nonlinearity to the feature map, improve the accuracy of the model, and also improve the generalization ability of the model.

[0130] It can be seen that this optional embodiment can input the target facial image into a face detection model including multiple feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; perform feature fusion on the corresponding shallow feature map and deep feature map to obtain the final feature map corresponding to the target facial image, which is beneficial to improving the training accuracy of the face detection model through feature extraction of the feature extraction unit and subsequent feature fusion, and is also beneficial to reducing the amount of model calculation and speeding up processing speed.

[0131] In yet another optional embodiment, for any target face image in the above steps, feature fusion is performed on the feature map corresponding to the target face image to obtain a final feature map corresponding to the target face image, which may include:

[0132] For any target face image, the deep feature map corresponding to the target face image extracted by the feature extraction unit is input into the SPPF layer for processing to obtain a processed deep feature map corresponding to the target face image;

[0133] For any target face image, the processed deep feature map corresponding to the target face image is feature fused with the shallow feature map corresponding to the target face image to obtain a final feature map corresponding to the target face image.

[0134] In this optional embodiment, the SPPF layer (spatial pyramid pooling fast layer) is used to allow the model to process input images of different sizes, and further use the feature map generated by the awFace network model to build a feature pyramid; further, global average pooling can be directly used for the last convolutional layer. Global average pooling is only related to the number of channels and can also achieve the purpose of fixing the input size.

[0135] It can be seen that this optional embodiment can input the deep feature map corresponding to the target face image extracted by the feature extraction unit into the SPPF layer for processing, obtain the processed deep feature map corresponding to the target face image, and perform feature fusion of the processed deep feature map with the shallow feature map to obtain the final feature map; processing the image based on the SPPF layer and fusing the processed deep feature map with the shallow feature map is conducive to enriching the information of the feature map, while improving the adaptability of the model to the input size and enhancing the robustness to scale changes.

[0136] In yet another optional embodiment, for any target facial image in the above steps, feature fusion of the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image may include:

[0137] For any target face image, among the multiple feature maps corresponding to the target face image, any feature map whose size is larger than that of the deep feature map and smaller than that of the shallow feature map is determined as the middle feature map corresponding to the target face image;

[0138] For any target face image, the processed deep feature map corresponding to the target face image is spliced ​​with the middle feature map corresponding to the target face image to obtain a first spliced ​​feature map corresponding to the target face image;

[0139] Splicing the first splicing feature map corresponding to the target face image with the shallow feature map corresponding to the target face image to obtain a second splicing feature map corresponding to the target face image;

[0140] Adding the first spliced ​​feature map corresponding to the target face image and the second spliced ​​feature map corresponding to the target face image element by element to obtain a first added feature map corresponding to the target face image;

[0141] The first added feature map corresponding to the target facial image and the processed deep feature map corresponding to the target facial image are added element by element to obtain a final feature map corresponding to the target facial image.

[0142] In this optional embodiment, feature maps at different levels in the feature pyramid are spliced ​​together, and feature maps at different layers contain different semantic information. Deep feature maps retain more semantic information, which is conducive to the overall summary of the image, such as whether a face exists; while shallow feature maps retain more spatial information, which is conducive to the determination of the spatial position of image targets, such as the position of a face. Therefore, the second spliced ​​feature map can obtain deep semantic information and shallow spatial information at the same time; by splicing them together, the network can better recognize faces of different sizes and postures while retaining key detail information.

[0143] In this optional embodiment, element-by-element addition can be performed by ADD, which refers to the direct addition of tensors and does not expand the dimension. The ADD process can fuse features from different network layers, allowing the model to learn high-level semantic information of deep feature maps and detailed spatial information of shallow feature maps.

[0144] In this optional embodiment, depthwise separable convolution may be used instead of ordinary convolution to further reduce computational overhead.

[0145] It can be seen that this optional embodiment can determine the middle-level feature map corresponding to the target facial image, splice the processed deep feature map to obtain a first spliced ​​feature map, splice the first spliced ​​feature map with the shallow feature map to obtain a second spliced ​​feature map, add the first spliced ​​feature map and the second spliced ​​feature map element by element to obtain a first added feature map, and add the first added feature map and the deep feature map element by element to obtain a final feature map; by combining deep semantic information and shallow spatial information, it is beneficial to improve the accuracy of recognizing faces of different sizes and postures, and thus to improve the accuracy of the model; in addition, deep separable convolution is used instead of ordinary convolution during training, and fully connected layers and repeated pooling operations are not used, which is beneficial to reducing the number of parameters of the network model.

[0146] In yet another optional embodiment, the loss parameter of the face detection model after the current training in the above step is determined based on at least one of the positioning loss parameter of the face detection model after the current training, the target confidence loss parameter of the face detection model after the current training, and the classification loss parameter of the face detection model after the current training;

[0147] The calculation method of the positioning loss parameter of the face detection model after this training is as follows:

[0148]

[0149] Among them, L CIoU (b,b gt ) represents the positioning loss parameter of the face detection model after this training, b represents the predicted face bounding box corresponding to the target face image predicted by the face detection model after this training, and b gt represents the real face bounding box annotated by the target face image, CIOU represents the complete intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, IOU represents the intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, d u Represents the distance between the center point of the predicted face bounding box predicted by the face detection model after this training and the center point of the real face bounding box, dc represents the maximum diagonal distance between the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, v represents the aspect ratio coefficient between the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, arctan represents the inverse tangent function, and w gt Represents the width of the real face bounding box, h gt represents the height of the real face bounding box, b w represents the width of the predicted face bounding box predicted by the face detection model after this training, b h Represents the height of the predicted face bounding box predicted by the face detection model after this training;

[0150] The target confidence loss parameter of the face detection model after this training is calculated as follows:

[0151]

[0152] Among them, L obj (p o ,p iou ) represents the target confidence loss parameter of the face detection model after this training; p o represents the probability of a face being contained in the predicted face bounding box predicted by the face detection model after this training, p iou represents the IOU confidence of the predicted face bounding box predicted by the face detection model after this training, w obj Represents the representation parameters corresponding to the target confidence loss parameters of the face detection model after this training; Indicates that when calculating the target confidence loss parameter of the trained face detection model based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1];

[0153] The classification loss parameters of the face detection model after this training are calculated as follows:

[0154]

[0155] Among them, L cls (c p ,c gt ) represents the classification loss parameter of the face detection model after this training; c p represents the probability that the face detection model after this training predicts the category accurately, c gt represents the true category label of the target face image, w cls Represents the representation parameters corresponding to the classification loss parameters of the face detection model after this training; Indicates that when calculating the classification loss parameters of the trained face detection model based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1].

[0156] In this optional embodiment, the loss parameter is determined based on the positioning loss parameter, the target confidence loss parameter and the classification loss parameter, and is calculated as follows:

[0157]

[0158] Among them, K, S, and B are the number of output feature maps, each grid cell, and multiple anchor boxes on each grid cell, respectively. box is the weight corresponding to the positioning loss parameter, α obj is the weight corresponding to the target confidence loss parameter, α cls is the weight corresponding to the classification loss parameter, Indicates that for the final feature map of the k-th output, it is determined whether the i-th cell and the j-th anchor box are positive samples (a positive sample refers to an anchor whose IoU value exceeds a predetermined threshold). If it is a positive sample, the value is 1, otherwise it is 0; t p and t gt are the predicted vector and the ground-truth vector respectively. K represents the number of feature maps output. Represents the balancing weight used to balance the final feature maps of different scales, and i and j represent the coordinate index on the final feature map. It can be understood that the face detection model first divides the input image into grids of fixed dimensions, and each grid cell is responsible for detecting the target (face) that falls within its area. Multiple anchor boxes are predefined on each cell, and these anchor boxes have different shapes and sizes to adapt to the detection of faces of different sizes and proportions. The model outputs a prediction vector for each anchor box, which contains the position, size, confidence (i.e., the probability of whether the box contains a face) and classification probability of the bounding box. These prediction vectors are compared with the true annotation vector (ground-truth vector) corresponding to each anchor box to calculate the combined loss function, which consists of classification loss (measuring the error between the predicted category and the actual category), localization loss (measuring the difference between the predicted bounding box and the true bounding box) and confidence loss (measuring the error between the confidence that the predicted bounding box contains the target and the actual situation). By minimizing this combined loss during the training process, the model learns how to accurately identify and locate faces in images, and how to accurately evaluate its own prediction confidence.

[0159] It can be seen that this optional embodiment can determine the loss parameter based on at least one of the positioning loss parameter, the target confidence loss parameter and the classification loss parameter, which is conducive to improving the accuracy of calculating whether the loss parameter meets the loss parameter conditions during the model training process, thereby improving the accuracy of the model training and thus improving the accuracy of the face detection model.

[0160] In an optional embodiment, in the above step 103, the face detection model after the current training is trained based on the target face images in the target face image data set that do not participate in the model training and the model weights of the face detection model after the current training, until the loss parameter of the face detection model after the current training meets the loss parameter condition matched therewith, in step 104, when it is judged that the model accuracy meets the model accuracy condition matched therewith, before determining that the face detection model to be tested is the trained face detection model, the method may also include:

[0161] The trained face detection model is tested using the target test set, and the model performance is quantitatively evaluated from multiple dimensions based on the test results to obtain the quantitative evaluation results of the trained face detection model;

[0162] In this optional embodiment, the target test set may be a WiderFace test set, and the quantitative evaluation of the model performance from multiple dimensions may be a quantitative evaluation of the model performance from three dimensions: easy, moderate, and hard. After actual testing, the results of the trained face detection model on the WiderFace test set are easy83.3%, moderate75.9%, and hard44.4% at an input resolution of 320*320, and easy88.0%, moderate82.7%, and hard61.8% at an input resolution of 480*480.

[0163] It can be seen that this optional embodiment can use the target test set to test the model and perform quantitative evaluation based on the test results, which is conducive to improving the understanding of the training results of the face detection model.

[0164] Embodiment 2

[0165] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another method for determining a face detection model for an edge device disclosed in an embodiment of the present invention. Figure 2 The method shown can be applied to application scenarios for training and determining face detection models for edge devices, such as model construction for specific face recognition, security systems, etc., which is not limited in the embodiments of the present invention. Figure 2As shown, the method for determining the face detection model for the edge device may include the following operations:

[0166] 201. Determine at least one target initial face image from an initial face image data set; for each target initial face image, determine a plurality of source images corresponding to the target initial face image;

[0167] In an embodiment of the present invention, the initial face image data set can be based on the open source face data set of WiderFace, adding some data of CelebA-Mask, randomly selecting 10,000 of the 220,000 images of the CelebA-Mask data set, and converting the data set label format to meet the requirements of algorithm training; converting the data set label format can be converting face detection related labels into txt format; face detection related labels include face, face frame coordinates and face key point coordinate labels; image content includes faces in various postures, and scenes with face occlusion such as wearing glasses, side faces, and masks; the multiple source images at least include the target initial face image or the augmented face image of the target initial face image after data augmentation; data augmentation operations include flipping, scaling, color gamut changes, etc.;

[0168] 202. For each target initial face image, multiple corresponding source images are spliced ​​in a minimum connection manner to obtain a combined image corresponding to the target initial face image;

[0169] In the embodiment of the present invention, the minimum connection mode means that when stitching multiple source images, the nearest edges between any two adjacent source images completely overlap or partially overlap, or only one vertex between any two adjacent source images overlaps;

[0170] 203. Setting a cropping frame corresponding to the target initial face image according to a preset size;

[0171] 204. For each target initial face image, select a target point on its corresponding combined image;

[0172] 205. For each target initial face image, based on the corresponding cropping frame, crop the corresponding combined image, and complete the cropped combined image based on the target point to obtain an intermediate face image corresponding to the target initial face image;

[0173] In the embodiment of the present invention, the data augmentation operation of each image is random, and the sizes after augmentation are different. After splicing them to corresponding positions to obtain a combined image, it is cropped, and the cropped combined image is completed based on the target point. The random point can be used as the center point of the output image, and the position of the edge of the final output image is determined according to the surrounding random points. Finally, the cropped image is set to 640*640, and the insufficient part is completed with pixels; the obtained intermediate face image contains the image of the face position and label information; optionally, the input can also be directly resized to directly achieve the purpose of fixing the input size;

[0174] 206. For all the first face images in the first face image data set, select some of the first face images and perform image transformation operations thereon to obtain a target face image data set;

[0175] In an embodiment of the present invention, the first face image dataset includes all target initial face images and all intermediate face images; performing an image transformation operation may be performing a copy operation and a Random affine transformation operation on a random image, and the operations include copy and paste, random rotation, translation, scaling, shearing and the like; after performing image transformation to obtain a target face image dataset after data enhancement, the target face image dataset is sent to the awFace network for random processing and feature extraction.

[0176] 207. Input the target face image in the target face image data set into the face detection model to be trained to obtain the loss parameter and model weight of the face detection model after the training;

[0177] In an embodiment of the present invention, the target face image data set includes target face images used for model training of a face detection model; the face detection model to be trained has corresponding initial model weights.

[0178] 208. When it is determined that the loss parameter of the face detection model after the current training does not satisfy the loss parameter condition matched therewith, the model weight of the face detection model after the previous training is updated based on the loss parameter of the face detection model after the current training to obtain the model weight of the face detection model after the current training;

[0179] 209. The face detection model after this training is trained based on the target face images in the target face image data set that are not involved in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training satisfies the loss parameter condition matched therewith;

[0180] 210. Perform a scene test based on the model weight corresponding to the face detection model to be tested that meets the loss parameter condition matched therewith, and obtain the model accuracy of the face detection model to be tested;

[0181] 211. When it is determined that the model accuracy meets the matching model accuracy condition, the face detection model to be tested is determined to be a trained face detection model.

[0182] In the embodiment of the present invention, for the description of steps 207-211, please refer to the detailed description of steps 101-105 in the first embodiment, which will not be repeated here.

[0183] It can be seen that the method described in the embodiment of the present invention can determine multiple source images corresponding to at least one target initial face image. For each target initial face image, the corresponding multiple source images are spliced ​​in a minimum connection manner to obtain a combined image; for each target initial face image, a target point is selected on the corresponding combined image, and the corresponding combined image is cropped based on the corresponding cropping frame, and the cropped combined image is complemented based on the target point to obtain an intermediate face image corresponding to the target initial face image; for all first face images in the first face image data set, some of the first face images are selected to perform image transformation operations; through image combination, cropping, complementation, transformation and other operations, it is beneficial to realize the augmentation of training data, enhance the richness of images used for training, and thereby improve the robustness of face detection model training.

[0184] Embodiment 3

[0185] See also Figure 3 , Figure 3 : is a schematic diagram of a structure of a device for determining a face detection model for an edge device disclosed in an embodiment of the present invention. Figure 3 The device shown can be applied to application scenarios for training and determining face detection models for edge devices, such as model construction for specific face recognition, security systems, etc., which is not limited in the embodiments of the present invention. Figure 3 As shown, the device for determining the face detection model for the edge device may include:

[0186] The first training module 301 is used to input the target face image in the target face image data set into the face detection model to be trained, and obtain the loss parameter and model weight of the face detection model after this training; the target face image data set includes the target face image for model training of the face detection model; the face detection model to be trained has a corresponding initial model weight;

[0187] An updating module 302 is used to update the model weight of the face detection model after the previous training based on the loss parameter of the face detection model after the current training, when it is determined that the loss parameter of the face detection model after the current training does not meet the loss parameter condition matched therewith, to obtain the model weight of the face detection model after the current training;

[0188] The second training module 303 is used to train the face detection model after the current training based on the target face images in the target face image data set that have not participated in the model training and the model weights of the face detection model after the current training, until the loss parameter of the face detection model after the current training meets the loss parameter condition matched therewith;

[0189] A testing module 304 is used to perform a scene test based on a model weight corresponding to the face detection model to be tested that meets the loss parameter condition matched therewith, so as to obtain a model accuracy rate of the face detection model to be tested;

[0190] The first determination module 305 is used to determine that the face detection model to be tested is a trained face detection model when it is determined that the model accuracy meets the model accuracy condition matched therewith.

[0191] It can be seen that the device described in the embodiment of the present invention inputs the target face image into the face detection model to be trained for training, and obtains the loss parameter and model weight of the model after this training; when the loss parameter of the model after this training does not meet the loss parameter condition, the model weight of the model after the previous training is updated; the model after this training is trained based on the target face image that did not participate in the model training and the model weight of the model after this training until the loss parameter meets the condition; based on the satisfied model to be tested, the scene test is performed to obtain the model accuracy; when it is judged that the model accuracy meets the condition, the model to be tested is determined to be a trained face detection model. It can be seen that the present invention can improve the accuracy of the face detection model, which is conducive to improving the accuracy of face recognition; and in the process of training the face detection model, there is no need to use a fully connected layer and repeatedly perform pooling operations on the model, which reduces the training parameters of the model, while ensuring the recognition accuracy of the model, improving the efficiency of model training and obtaining a lightweight model, thereby achieving a balance between model recognition accuracy and network scale.

[0192] In an optional embodiment, the first training module 301 inputs the target face image in the target face image data set into the face detection model to be trained for training, and the specific method of obtaining the loss parameter and model weight of the face detection model after this training includes:

[0193] Inputting the target face image in the target face image data set into the face detection model to be trained for training, and obtaining a final feature map of the target face image in the target face image data set;

[0194] Based on the final feature map of each target face image, determine all predicted face bounding boxes of the target face image;

[0195] Based on all real face bounding boxes annotated for each target face image and all predicted face bounding boxes for the target face image, the loss parameters and model weights of the trained face detection model are obtained.

[0196] It can be seen that the device described in the embodiment of the present invention can input the target face image into the face detection model to be trained for training, obtain the final feature map of the target face image in the target face image data set, and determine the predicted face bounding box based on the final feature map. The loss parameters and model weights of the face detection model after this training are obtained in combination with the real face bounding box, which is beneficial to improving the training accuracy of the face detection model, and further ensuring the model's recognition accuracy of the face while reducing the model parameters.

[0197] In another optional embodiment, Figure 4 As shown, the device may also include:

[0198] The second determination module 306 is used to input the target face image in the target face image data set into the face detection model to be trained in the first training module 301, and before obtaining the loss parameter and model weight of the face detection model after this training, determine at least one target initial face image from the initial face image data set; for each target initial face image, determine multiple source images corresponding to the target initial face image; the multiple source images at least include the target initial face image or the augmented face image after the target initial face image is augmented by data;

[0199] The stitching module 307 is used to stitch the corresponding multiple source images of each target initial face image in a minimum connection manner to obtain a combined image corresponding to the target initial face image; the minimum connection manner means that when stitching the multiple source images, the nearest edges between any two adjacent source images completely overlap or partially overlap, or only one vertex between any two adjacent source images overlaps;

[0200] A setting module 308, used to set a cropping frame corresponding to the target initial face image according to a preset size;

[0201] A selection module 309 is used to select a target point on the corresponding combined image for each target initial face image;

[0202] The cropping module 310 is used to crop the corresponding combined image of each target initial face image based on the corresponding cropping frame, and complete the cropped combined image based on the target point to obtain an intermediate face image corresponding to the target initial face image;

[0203] The transformation module 311 is used to select some of the first face images from all the first face images in the first face image data set and perform image transformation operations to obtain a target face image data set; the first face image data set includes all target initial face images and all intermediate face images.

[0204] It can be seen that the device described in the embodiment of the present invention is capable of determining multiple source images corresponding to at least one target initial face image, and for each target initial face image, the multiple corresponding source images are spliced ​​in a minimum connection manner to obtain a combined image; for each target initial face image, a target point is selected on the corresponding combined image, and the corresponding combined image is cropped based on the corresponding cropping frame, and the cropped combined image is complemented based on the target point to obtain an intermediate face image corresponding to the target initial face image; for all first face images in the first face image data set, some of the first face images are selected to perform image transformation operations; through image combination, cropping, complementation, transformation and other operations, it is beneficial to realize the augmentation of training data, enhance the richness of images used for training, and thereby improve the robustness of face detection model training.

[0205] In another optional embodiment, the first training module 301 inputs the target face image in the target face image data set into the face detection model to be trained for training, and the specific method of obtaining the final feature map of the target face image in the target face image data set includes:

[0206] Input the target face image in the target face image data set into a face detection model including multiple feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; when the feature extraction unit is a deep feature extraction unit, the feature map corresponding to the feature extraction unit is a deep feature map; when the feature extraction unit is a shallow feature extraction unit, the feature map corresponding to the feature extraction unit is a shallow feature map;

[0207] For any target face image, the feature map corresponding to the target face image is feature fused to obtain a final feature map corresponding to the target face image; the feature map corresponding to the target face image includes a shallow feature map and a deep feature map.

[0208] It can be seen that the device described in the embodiment of the present invention can input the target facial image into a face detection model including multiple feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; perform feature fusion on the corresponding shallow feature map and the deep feature map to obtain the final feature map corresponding to the target facial image, which is beneficial to improving the training accuracy of the face detection model through feature extraction of the feature extraction unit and subsequent feature fusion, and is also beneficial to reducing the amount of model calculation and speeding up the processing speed.

[0209] In another optional embodiment, the first training module 301 performs feature fusion on the feature map corresponding to any target face image to obtain the final feature map corresponding to the target face image in a specific manner including:

[0210] For any target face image, the deep feature map corresponding to the target face image extracted by the feature extraction unit is input into the SPPF layer for processing to obtain a processed deep feature map corresponding to the target face image;

[0211] For any target face image, the processed deep feature map corresponding to the target face image is feature fused with the shallow feature map corresponding to the target face image to obtain a final feature map corresponding to the target face image.

[0212] It can be seen that the device described in the embodiment of the present invention can input the deep feature map corresponding to the target face image extracted by the feature extraction unit into the SPPF layer for processing, obtain the processed deep feature map corresponding to the target face image, and perform feature fusion of the processed deep feature map with the shallow feature map to obtain the final feature map; processing the image based on the SPPF layer and fusing the processed deep feature map with the shallow feature map is conducive to enriching the information of the feature map, while improving the adaptability of the model to the input size and enhancing the robustness to scale changes.

[0213] In another optional embodiment, the first training module 301 performs feature fusion on any target face image by performing feature fusion on the processed deep feature map corresponding to the target face image and the shallow feature map corresponding to the target face image to obtain the final feature map corresponding to the target face image. The specific method includes:

[0214] For any target face image, among the multiple feature maps corresponding to the target face image, any feature map whose size is larger than that of the deep feature map and smaller than that of the shallow feature map is determined as the middle feature map corresponding to the target face image;

[0215] For any target face image, the processed deep feature map corresponding to the target face image is spliced ​​with the middle feature map corresponding to the target face image to obtain a first spliced ​​feature map corresponding to the target face image;

[0216] Splicing the first splicing feature map corresponding to the target face image with the shallow feature map corresponding to the target face image to obtain a second splicing feature map corresponding to the target face image;

[0217] Adding the first spliced ​​feature map corresponding to the target face image and the second spliced ​​feature map corresponding to the target face image element by element to obtain a first added feature map corresponding to the target face image;

[0218] The first added feature map corresponding to the target facial image and the processed deep feature map corresponding to the target facial image are added element by element to obtain a final feature map corresponding to the target facial image.

[0219] It can be seen that the device described in the embodiment of the present invention can determine the middle-level feature map corresponding to the target facial image, splice the processed deep feature map to obtain a first spliced ​​feature map, splice the first spliced ​​feature map with the shallow feature map to obtain a second spliced ​​feature map, add the first spliced ​​feature map and the second spliced ​​feature map element by element to obtain a first added feature map, and add the first added feature map and the deep feature map element by element to obtain a final feature map; by combining deep semantic information and shallow spatial information, it is beneficial to improve the accuracy of recognizing faces of different sizes and postures, and thus to improve the accuracy of the model; in addition, deep separable convolution is used instead of ordinary convolution during training, and fully connected layers and repeated pooling operations are not used, which is beneficial to reducing the number of parameters of the network model.

[0220] In yet another optional embodiment, the loss parameter of the face detection model after the current training is determined based on at least one of the positioning loss parameter of the face detection model after the current training, the target confidence loss parameter of the face detection model after the current training, and the classification loss parameter of the face detection model after the current training;

[0221] The calculation method of the positioning loss parameter of the face detection model after this training is as follows:

[0222]

[0223] Among them, L CIoU (b,b gt ) represents the positioning loss parameter of the face detection model after this training, b represents the predicted face bounding box corresponding to the target face image predicted by the face detection model after this training, and b gtrepresents the real face bounding box annotated by the target face image, CIOU represents the complete intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, IOU represents the intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, d u Represents the distance between the center point of the predicted face bounding box predicted by the face detection model after this training and the center point of the real face bounding box, d c represents the maximum diagonal distance between the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, v represents the aspect ratio coefficient between the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, arctan represents the inverse tangent function, and w gt Represents the width of the real face bounding box, h gt represents the height of the real face bounding box, b w represents the width of the predicted face bounding box predicted by the face detection model after this training, b h Represents the height of the predicted face bounding box predicted by the face detection model after this training;

[0224] The target confidence loss parameter of the face detection model after this training is calculated as follows:

[0225]

[0226] Among them, L obj (p o ,p iou ) represents the target confidence loss parameter of the face detection model after this training; p o represents the probability of a face being contained in the predicted face bounding box predicted by the face detection model after this training, p iou represents the IOU confidence of the predicted face bounding box predicted by the face detection model after this training, w obj Represents the representation parameters corresponding to the target confidence loss parameters of the face detection model after this training; Indicates that when calculating the target confidence loss parameter of the trained face detection model based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1];

[0227] The classification loss parameters of the face detection model after this training are calculated as follows:

[0228]

[0229] Among them, L cls (c p ,cgt ) represents the classification loss parameter of the face detection model after this training; c p represents the probability that the face detection model after this training predicts the category accurately, c gt represents the true category label of the target face image, w cls Represents the representation parameters corresponding to the classification loss parameters of the face detection model after this training; Indicates that when calculating the classification loss parameters of the trained face detection model based on binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0,1].

[0230] It can be seen that the device described in the embodiment of the present invention can determine the loss parameter based on at least one of the positioning loss parameter, the target confidence loss parameter and the classification loss parameter, which is beneficial to improve the accuracy of calculating whether the loss parameter meets the loss parameter conditions during the model training process, thereby improving the accuracy of the model training and thus improving the accuracy of the face detection model.

[0231] Embodiment 4

[0232] See also Figure 5 , Figure 5 is a schematic diagram of the structure of another device for determining a face detection model for an edge device disclosed in an embodiment of the present invention. Figure 5 The described device can be applied in an application server. Figure 5 As shown, the device may include:

[0233] A memory 401 storing executable program codes;

[0234] a processor 402 coupled to the memory 401;

[0235] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the method for determining the face detection model for an edge device described in Embodiment 1 or Embodiment 2 of the present invention.

[0236] Embodiment 5

[0237] An embodiment of the present invention discloses a computer-storable medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the method for determining a face detection model for an edge device described in Embodiment 1 of the present invention or Embodiment 2 of the present invention.

[0238] Embodiment 6

[0239] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the method for determining a face detection model for an edge device described in Embodiment 1 or Embodiment 2 of the present invention.

[0240] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0241] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0242] Finally, it should be noted that the method and device for determining the face detection model for edge devices disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical schemes described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical schemes from the spirit and scope of the technical schemes of the embodiments of the present invention.

Claims

1. A method for determining a face detection model for an edge device, characterized in that: include: Inputting the target face images in the target face image data set into the face detection model to be trained, and obtaining the loss parameters and model weights of the face detection model after the training; the target face image data set includes the target face images for model training of the face detection model; the face detection model to be trained has a corresponding initial model weight; When it is determined that the loss parameter of the face detection model after the current training does not satisfy the loss parameter condition matched therewith, the model weight of the face detection model after the previous training is updated based on the loss parameter of the face detection model after the current training to obtain the model weight of the face detection model after the current training; The face detection model after this training is trained based on the target face images in the target face image data set that are not involved in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training meets the loss parameter condition matched therewith; Performing a scene test based on the model weight corresponding to the face detection model to be tested that meets the loss parameter condition matched therewith, to obtain the model accuracy of the face detection model to be tested; When it is determined that the model accuracy meets the model accuracy condition matched therewith, it is determined that the face detection model to be tested is the trained face detection model.

2. The method for determining a face detection model for an edge device according to claim 1, characterized in that: The target face image in the target face image data set is input into the face detection model to be trained for training, and the loss parameter and model weight of the face detection model after the training are obtained, including: Inputting the target face image in the target face image data set into the face detection model to be trained for training, and obtaining a final feature map of the target face image in the target face image data set; Based on the final feature map of each target face image, determining all predicted face bounding boxes of the target face image; Based on all the real face bounding boxes annotated in each of the target face images and all the predicted face bounding boxes of the target face image, the loss parameters and model weights of the face detection model after this training are obtained.

3. The method for determining a face detection model for an edge device according to claim 1, characterized in that: Before inputting the target face image in the target face image data set into the face detection model to be trained for training and obtaining the loss parameter and model weight of the face detection model after the training, the method further includes: Determine at least one target initial face image from the initial face image data set; for each of the target initial face images, determine a plurality of source images corresponding to the target initial face image; the plurality of source images at least include the target initial face image or an augmented face image of the target initial face image after data augmentation; For each target initial face image, the corresponding multiple source images are spliced ​​in a minimum connection manner to obtain a combined image corresponding to the target initial face image; the minimum connection manner means that when splicing the multiple source images, the nearest edges between any two adjacent source images completely overlap or partially overlap, or only one vertex between any two adjacent source images overlaps; Setting a cropping frame corresponding to the target initial face image according to a preset size; For each of the target initial face images, selecting a target point on its corresponding combined image; For each of the target initial face images, based on the corresponding cropping frame, the corresponding combined image is cropped, and based on the target point, the cropped combined image is completed to obtain an intermediate face image corresponding to the target initial face image; For all the first face images in the first face image data set, some of the first face images are selected to perform image transformation operations to obtain a target face image data set; the first face image data set includes all the target initial face images and all the intermediate face images.

4. The method for determining a face detection model for an edge device according to claim 2, characterized in that: The step of inputting the target face image in the target face image data set into the face detection model to be trained to obtain a final feature map of the target face image in the target face image data set includes: Input the target face image in the target face image data set into a face detection model including a plurality of feature extraction units for feature extraction, and obtain a feature map corresponding to each feature extraction unit; when the feature extraction unit is a deep feature extraction unit, the feature map corresponding to the feature extraction unit is a deep feature map; when the feature extraction unit is a shallow feature extraction unit, the feature map corresponding to the feature extraction unit is a shallow feature map; For any of the target facial images, feature fusion is performed on the feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image; the feature map corresponding to the target facial image includes the shallow feature map and the deep feature map.

5. The method for determining a face detection model for an edge device according to claim 4, characterized in that: For any of the target face images, feature fusion is performed on the feature map corresponding to the target face image to obtain a final feature map corresponding to the target face image, including: For any of the target facial images, the deep feature map corresponding to the target facial image extracted by the feature extraction unit is input into the SPPF layer for processing to obtain a processed deep feature map corresponding to the target facial image; For any of the target facial images, feature fusion is performed on the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image.

6. The method for determining a face detection model for an edge device according to claim 5, characterized in that: For any of the target facial images, feature fusion is performed on the processed deep feature map corresponding to the target facial image and the shallow feature map corresponding to the target facial image to obtain a final feature map corresponding to the target facial image, including: For any of the target facial images, among the multiple feature maps corresponding to the target facial image, any feature map whose size is larger than that of the deep feature map and smaller than that of the shallow feature map is determined as a middle-layer feature map corresponding to the target facial image; For any of the target facial images, splicing the processed deep feature map corresponding to the target facial image with the middle feature map corresponding to the target facial image to obtain a first spliced ​​feature map corresponding to the target facial image; Splicing the first splicing feature map corresponding to the target face image with the shallow feature map corresponding to the target face image to obtain a second splicing feature map corresponding to the target face image; Adding the first splicing feature map corresponding to the target facial image and the second splicing feature map corresponding to the target facial image element by element to obtain a first added feature map corresponding to the target facial image; The first added feature map corresponding to the target facial image and the processed deep feature map corresponding to the target facial image are added element by element to obtain a final feature map corresponding to the target facial image.

7. The method for determining a face detection model for an edge device according to claim 2, characterized in that: The loss parameter of the face detection model after the current training is determined based on at least one of the positioning loss parameter of the face detection model after the current training, the target confidence loss parameter of the face detection model after the current training, and the classification loss parameter of the face detection model after the current training; The calculation method of the positioning loss parameter of the face detection model after this training is as follows: Among them, L CIoU (b,b gt ) represents the positioning loss parameter of the face detection model after this training, b represents the predicted face bounding box corresponding to the target face image predicted by the face detection model after this training, and b gt represents the real face bounding box annotated by the target face image, CIOU represents the complete intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, IOU represents the intersection-over-union ratio of the predicted face bounding box predicted by the face detection model after this training and the real face bounding box, d u represents the distance between the center point of the predicted face bounding box predicted by the face detection model after this training and the center point of the real face bounding box, d c represents the maximum value of the diagonal distance between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, v represents the aspect ratio coefficient between the predicted face bounding box and the real face bounding box predicted by the face detection model after this training, arctan represents the inverse tangent function, and w gt represents the width of the real face bounding box, h gt represents the height of the real face bounding box, b w represents the width of the predicted face bounding box predicted by the face detection model after this training, b h Represents the height of the predicted face bounding box predicted by the face detection model after this training; The target confidence loss parameter of the face detection model after this training is calculated as follows: Among them, L obj (p o , p iou ) represents the target confidence loss parameter of the face detection model after this training; p o represents the probability that the predicted face bounding box contains a face predicted by the face detection model after this training, pio u represents the IOU confidence of the predicted face bounding box predicted by the face detection model after this training, w obj Represents the representation parameter corresponding to the target confidence loss parameter of the face detection model after this training; represents that when calculating the target confidence loss parameter of the face detection model after the current training based on the binary cross entropy, the output probability is limited to the interval [0, 1] using the Sigmoid activation function; The classification loss parameters of the face detection model after this training are calculated as follows: Among them, L cls (c p , c gt ) represents the classification loss parameter of the face detection model after this training; c p represents the probability that the category predicted by the face detection model after this training is accurate, c gt represents the real category label of the target face image, w cls Represents the representation parameter corresponding to the classification loss parameter of the face detection model after this training; When calculating the classification loss parameter of the face detection model after the current training based on the binary cross entropy, the Sigmoid activation function is used to limit the output probability to the interval [0, 1].

8. A device for determining a face detection model for an edge device, characterized in that: The device comprises: The first training module is used to input the target face image in the target face image data set into the face detection model to be trained, and obtain the loss parameter and model weight of the face detection model after the training; the target face image data set includes the target face image for model training of the face detection model; the face detection model to be trained has a corresponding initial model weight; An updating module, configured to update the model weights of the face detection model after the previous training based on the loss parameters of the face detection model after the current training, when it is determined that the loss parameters of the face detection model after the current training do not meet the loss parameter conditions matched therewith, to obtain the model weights of the face detection model after the current training; A second training module is used to train the face detection model after this training based on the target face images in the target face image data set that have not participated in the model training and the model weights of the face detection model after this training, until the loss parameter of the face detection model after this training meets the loss parameter condition matched therewith; A testing module, used to perform a scene test based on a model weight corresponding to the face detection model to be tested that satisfies the loss parameter condition matched therewith, to obtain a model accuracy rate of the face detection model to be tested; The first determination module is used to determine that the face detection model to be tested is the trained face detection model when it is determined that the model accuracy meets the model accuracy condition matched therewith.

9. A device for determining a face detection model for an edge device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for determining a face detection model for an edge device as described in any one of claims 1-7.

10. A computer storable medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the method for determining a face detection model for an edge device as described in any one of claims 1-7.

Images