Image detection method and device, storage medium and electronic device

Abstract

This application discloses an image detection method, apparatus, storage medium, and electronic device. Relating to the field of artificial intelligence, the method includes: acquiring N candidate face detection boxes and N initial confidence scores detected in a target image, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes have a one-to-one correspondence with the N initial confidence scores; iteratively processing the N candidate face detection boxes until the set of retained candidate face detection boxes is empty; determining the M candidate face detection boxes as the results detected in the target image, and displaying the M candidate face detection boxes in the target image. This application solves the problem of low accuracy in image detection in related technologies.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to an image detection method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] Image detection is being applied in an increasing number of fields, such as face detection. In related technologies, face detection often involves using neural network models to detect faces in an image, and then discarding face detection boxes whose confidence scores do not meet a certain threshold.

[0003] This approach may result in the neural network model outputting face detection bounding boxes that do not accurately represent faces in the image. For example, if the confidence threshold is set too high, some bounding boxes for the same face may not be suppressed, leading to false positives. If the confidence threshold is set too low, if two face detection boxes are adjacent, the confidence of one of the adjacent bounding boxes may be lower than that of the other. In such cases, the bounding box with the lower confidence may also be suppressed, resulting in missed detections.

[0004] There is currently no effective solution to the problem of low accuracy in image detection in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide an image detection method, apparatus, storage medium, and electronic device to solve the problem of low accuracy in image detection in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, an image detection method is provided. The method includes: acquiring N candidate face detection boxes and N initial confidence scores detected in a target image, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence; iteratively processing the N candidate face detection boxes until the set of retained candidate face detection boxes is empty, wherein the iterative processing includes M rounds of processing, and a total of M candidate face detection boxes are extracted after the M rounds of processing, where M is a positive integer greater than or equal to 1, and the candidate face detection boxes extracted in the i-th round are extracted from the set of candidate face detection boxes retained in the (i-1)-th round based on the confidence score set updated in the (i-1)-th round. In the i-th round of processing, the candidate face detection boxes need to be extracted from the set of candidate face detection boxes retained in the (i-1)-th round. When deleting candidate face detection boxes, the deleted candidate face detection boxes are those determined based on the confidence set updated in the i-th round. The confidence set updated in the (i-1)-th round includes the confidence scores of each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round. When i = 1, the confidence set updated in the (i-1)-th round includes the N initial confidence scores, and the candidate face detection box set retained in the (i-1)-th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M. The M candidate face detection boxes are determined as the results detected in the target image, and the M candidate face detection boxes are displayed in the target image.

[0007] Optionally, the iterative processing of the N candidate face detection boxes until the set of retained candidate face detection boxes is empty includes: performing the i-th round of processing through the following steps: extracting the corresponding candidate face detection boxes from the set of candidate face detection boxes retained in the (i-1)-th round to obtain the set of candidate face detection boxes to be determined in the i-th round, wherein, in the confidence set updated in the (i-1)-th round, the confidence of the candidate face detection boxes extracted in the i-th round is the highest; determining each candidate face detection box in the set of candidate face detection boxes retained in the (i-1)-th round other than the candidate face detection boxes extracted in the i-th round. The intersection-union ratios (IU / U) between the candidate face detection boxes extracted in the i-th round and the i-th round are used to obtain the i-th round IU / U set. The confidence scores in the (i-1)-th round updated confidence score set, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, are updated using the i-th round IU / U set to obtain the i-th round updated confidence score set. In the i-th round updated confidence score set, the confidence scores that are less than a preset threshold are determined, and the candidate face detection boxes with confidence scores less than the preset threshold are deleted from the i-th round candidate face detection box set to be determined, thus obtaining the i-th round retained candidate face detection box set.

[0008] Optionally, updating the confidence scores in the confidence score set updated in the (i-1)th round (excluding the confidence scores of the candidate face detection boxes extracted in the i-th round) using the intersection-union ratio (IU) set of the i-th round to obtain the confidence score set updated in the i-th round includes: the candidate face detection boxes in the candidate face detection box set retained in the (i-1)th round (excluding the candidate face detection boxes extracted in the i-th round) include P candidate face detection boxes; the confidence scores in the confidence score set updated in the (i-1)th round (excluding the confidence scores of the candidate face detection boxes extracted in the i-th round) include P confidence scores; the P confidence scores include the confidence scores of each candidate face detection box in the P candidate face detection boxes updated in the (i-1)th round; and the intersection-union ratio (IU) set of the i-th round includes P intersection-union ratios, where the P intersection-union ratios are the scores of each candidate face detection box extracted in the i-th round compared with the confidence scores of the candidate face detection boxes extracted in the i-th round. In the case of the intersection-union ratio (IU) among the P candidate face detection boxes, the following steps are performed to update the j-th confidence score among the P confidence scores, where the j-th IU is the IU between the candidate face detection boxes extracted in the i-th round and the j-th candidate face detection box among the P candidate face detection boxes, and the j-th confidence score is the confidence score of the j-th candidate face detection box updated in the (i-1)-th round, P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: an update factor for the j-th confidence score is determined based on the j-th IU; and the confidence score of the j-th candidate face detection box updated in the i-th round is determined based on the update factor, the j-th confidence score, and the j-th IU.

[0009] Optionally, the update factor of the j-th confidence level is negatively correlated with the j-th cross-union ratio, wherein the larger the j-th cross-union ratio, the smaller the update factor of the j-th confidence level.

[0010] Optionally, determining the confidence level of the j-th candidate face detection box in the i-th round of updates based on the update factor of the j-th confidence level, the j-th confidence level, and the j-th intersection-union ratio (IUU) includes: obtaining a first product of the update factor of the j-th confidence level and the j-th confidence level, and obtaining a first index value using the first product as a first base and a preset weight as a first exponent; obtaining a second index value using the j-th IUU as a second base and the difference between the target value and the preset weight as a second exponent; and determining the confidence level of the j-th candidate face detection box in the i-th round of updates as the product of the first index value and the second index value.

[0011] Optionally, determining the intersection-union ratio (IU / R) between each candidate face detection box (excluding the candidate face detection boxes extracted in the i-1th round) retained in the candidate face detection box set and the candidate face detection boxes extracted in the i-th round includes: when the candidate face detection boxes (excluding the candidate face detection boxes extracted in the i-1th round) retained in the candidate face detection box set include P candidate face detection boxes, the following steps are performed to determine the j-th IU / R between the j-th candidate face detection box in the P candidate face detection boxes and the candidate face detection boxes extracted in the i-th round, where P is greater than or equal to 1. The j is a positive integer equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P; the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round are input into the target recognition model to obtain the j-th intersection-union ratio, wherein the target recognition model is a model obtained by training the initial recognition model to be trained using the sample face detection box set and the actual intersection-union ratio between each sample face detection box in the sample face detection box set and the sample face detection box with the highest confidence in the sample face detection box set, and each sample face detection box in the sample face detection box set has a corresponding confidence.

[0012] Optionally, before inputting the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into the target recognition model, the method further includes: performing the k-th round of training on the initial recognition model to be trained through the following steps, where k is a positive integer greater than or equal to 2: inputting the k-th group of sample face detection boxes used in the k-th round into the initial recognition model obtained from the (k-1)-th round of training to obtain the k-th group of predicted crossover ratios, wherein the set of sample face detection boxes includes the k-th group of sample face detection boxes, and the k-th group of sample face detection boxes includes Q k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k A positive integer greater than or equal to 2; the loss function for the k-th round is determined based on the actual cross-union ratio (CUN) of the k-th group used in the k-th round and the predicted CUN of the k-th group, wherein the actual CUN of the k-th group includes the Q... k-1 Sample face detection box and Q respectively k The actual crossover ratio between the face detection boxes with the highest confidence in each sample face detection box; if the loss function in the k-th round satisfies the preset convergence condition, the training of the initial recognition model to be trained ends, and the initial recognition model obtained in the (k-1)-th round of training is determined as the target recognition model; if the loss function in the k-th round does not satisfy the convergence condition, the parameters in the initial recognition model obtained in the (k-1)-th round of training are adjusted to obtain the initial recognition model obtained in the k-th round of training.

[0013] Optionally, determining the loss function for the k-th round based on the actual cross-union ratio (CUN) of the k-th group used in the k-th round and the predicted CUN of the k-th group includes: obtaining a set of cross-entropy loss values ​​based on the first set of actual CUN and the second set of actual CUN, wherein the first set of actual CUN includes W actual CUNs, the second set of actual CUNs includes W actual CUNs, the set of cross-entropy loss values ​​includes W cross-entropy loss values, and the i-th actual CUN in the first set of actual CUNs is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence in the second set of face detection boxes is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual cross-entropy ratio (COP) between the face detection boxes with the highest confidence among the sample face detection boxes is given by the W cross-entropy loss values. The i-th cross-entropy loss value is obtained by performing cross-entropy loss calculation on the i-th actual cross-entropy ratio in the first group and the i-th actual cross-entropy ratio in the second group. W equals Q. k -1, where i is greater than or equal to 1 and less than or equal to Q. k -1 is a positive integer; the loss function of the k-th round is determined as the average of the W actual cross-union ratios.

[0014] To achieve the above objectives, according to another aspect of this application, an image detection apparatus is provided. The apparatus includes: a first input module, configured to acquire N candidate face detection boxes detected in a target image and N initial confidence scores, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence; and an iterative processing module, configured to iteratively process the N candidate face detection boxes until the set of retained candidate face detection boxes is empty, wherein the iterative processing includes M rounds of processing, after which a total of M candidate face detection boxes are extracted, where M is a positive integer greater than or equal to 1, and the candidate face detection boxes extracted in the i-th round are extracted from the set of candidate face detection boxes retained in the (i-1)-th round based on the confidence score set updated in the (i-1)-th round. In the i-th round of processing, it is necessary to extract candidate face detection boxes from the set of candidate face detection boxes retained in the (i-1)-th round. When a candidate face detection box is deleted from the detection box set, the deleted candidate face detection box is a candidate face detection box determined according to the confidence set updated in the i-th round. The confidence set updated in the (i-1)-th round includes the confidence of each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round. When i = 1, the confidence set updated in the (i-1)-th round includes the N initial confidences. The candidate face detection box set retained in the (i-1)-th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M. The processing module is used to determine the M candidate face detection boxes as the results detected in the target image and display the M candidate face detection boxes in the target image.

[0015] This application employs the following steps: Multiple iterations are performed on the N candidate face detection boxes output by the target neural network model. During the i-th iteration, candidate face detection boxes are extracted from the set of candidate face detection boxes retained in the (i-1)-th iteration based on the updated confidence set, until the retained set of candidate face detection boxes is empty. During the previous iteration, the confidence set required for the next iteration is updated, and the candidate face detection boxes in the set retained in the previous iteration are filtered based on the updated confidence set. This addresses the problem of low image detection accuracy in related technologies, thereby improving the accuracy of image detection. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart of an image detection method provided according to an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of a P-Net network structure according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of an R-Net network structure according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of an O-Net network structure according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of an image detection method according to an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of an optional target recognition model according to an embodiment of this application;

[0023] Figure 7 This is a schematic diagram illustrating face detection when the face is obscured, according to an embodiment of this application.

[0024] Figure 8 This is a schematic diagram of a small face detection according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the ROC curve on an FDDB test set according to an embodiment of this application;

[0026] Figure 10 This is a schematic diagram of an image detection apparatus according to an embodiment of this application;

[0027] Figure 11 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0032] The following describes this application in conjunction with the preferred implementation steps. Figure 1 This is a flowchart of an image detection method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0033] Step S101: Obtain N candidate face detection boxes and N initial confidence scores detected in the target image, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence.

[0034] Step S102: Iterate through the N candidate face detection boxes until the set of retained candidate face detection boxes is empty. The iterative process includes M rounds of processing, extracting a total of M candidate face detection boxes, where M is a positive integer greater than or equal to 1. The candidate face detection boxes extracted in the i-th round are obtained from the set of retained candidate face detection boxes in the (i-1)-th round based on the confidence set updated in the (i-1)-th round. In the i-th round, candidate face detection boxes need to be deleted from the set of retained candidate face detection boxes in the (i-1)-th round. In the case of [missing information], the deleted candidate face detection boxes are candidate face detection boxes determined according to the confidence set updated in the i-th round. The confidence set updated in the (i-1)-th round includes the confidence of each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round in the (i-1)-th round. When i = 1, the confidence set updated in the (i-1)-th round includes the N initial confidences, and the candidate face detection box set retained in the (i-1)-th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M.

[0035] Step S103: Determine the M candidate face detection boxes as the results detected in the target image, and display the M candidate face detection boxes in the target image.

[0036] In the technical solution provided in step S101 above, the target image can be input into the target neural network, but is not limited to this. The target neural network model can be used to detect faces in the target image and output possible face detection boxes and their corresponding confidence scores. For example, the target neural network model can be, but is not limited to, an MTCNN (Multi-task convolutional neural network) model, a CNN (Convolutional Neural Networks), an RNN (Recurrent Neural Network), or a DNN (Deep Neural Network), etc., or the target neural network model can include, but is not limited to, various sub-networks in the MTCNN model, CNN, RNN, or DNN, etc.

[0037] Optionally, in the image detection method provided in this application embodiment, some candidate face detection boxes detected by the target neural network model in the target image may not actually contain a face in the target image, or multiple candidate face detection boxes detected by the target neural network model in the target image may correspond to the same face in the target image. In such cases, the N candidate face detection boxes output by the target neural network model can be filtered based on, but is not limited to, the confidence level of each candidate face detection box, thereby improving the accuracy of the candidate face detection boxes output by the target neural network model.

[0038] Optionally, in the image detection method provided in this application embodiment, the structure of the target neural network model in this application embodiment can be explained and described using MTCNN as an example, but is not limited to. The MTCNN model includes three sub-networks: Proposal Network (P-Net), Refine Network (R-Net), and Output Network (O-Net). Each sub-neural network includes three tasks: face classification, bounding box regression, and keypoint localization. Through the three relatively shallow CNN network structures, redundant or erroneous face detection boxes are continuously removed, thereby effectively detecting faces and their five keypoints. The multi-task cascaded network MTCNN is a two-stage network and is a commonly used detection model for faces. The cascaded P-Net, R-Net, and O-Net networks form the MTCNN model.

[0039] In detail, P-Net is a Fully Connected Network (FCN). In FCN, there are no requirements on the size of the input. Therefore, P-Net, as an FCN, can accept input images of any size. Figure 2 This is a schematic diagram of a P-Net network structure according to an embodiment of this application, such as... Figure 2As shown, during training, the P-Net network input is 12×12 pixels. After several convolutional and pooling layers, a 1×1 feature map is obtained. Finally, after several 1×1×32 convolutions, a 1×1×2 face classification output, a 1×1×4 bounding box regression, and a 1×1×10 facial landmark location output are obtained. As the first sub-network of MTCNN, P-Net generates a large number of candidate detection boxes in this layer. Then, the non-maximum suppression (NMS) algorithm compares the intersection-over-union (IoU) of the candidate detection boxes and the ground truth boxes with a manually set threshold, eliminating unqualified prediction boxes and selecting qualified prediction boxes. These qualified prediction boxes are then input into R-Net.

[0040] Figure 3 This is a schematic diagram of an R-Net network structure according to an embodiment of this application, such as... Figure 3 As shown, the R-Net network structure consists of several convolutional layers and two pooling layers. Unlike the P-Net network, R-Net adds a 128-dimensional fully connected layer at the end of the network. The P-Net network scales its output to a size of 24×24 as the input to the R-Net network, and then uses NMS post-processing to pass the output as the input to the next sub-network, O-Net.

[0041] Figure 4 This is a schematic diagram of an O-Net network structure according to an embodiment of this application, such as... Figure 4 As shown, the network structure of O-Net is similar to that of R-Net. R-Net uses a 128-dimensional fully connected layer, while O-Net introduces a 256-dimensional fully connected layer. The image input size of the O-Net network is 48×48, which is obtained by scaling the output of the previous sub-network, R-Net. Similarly, the O-Net network also uses the NMS post-processing algorithm to accurately output face binary classification, face candidate boxes, and five face key points. Figure 2-3 It is an O-Net network structure.

[0042] In the technical solution provided in step S102 above, during the (i-1)th iteration, the confidence of each candidate face detection box in the candidate face detection box set retained in the (i-1)th iteration can be updated to obtain the updated confidence set for the (i-1)th iteration. During the (i)th iteration, candidate face detection boxes can be extracted from the candidate face detection box set retained in the (i-1)th iteration based on the updated confidence set for the (i-1)th iteration.

[0043] Optionally, in the image detection method provided in this application embodiment, in order to improve the accuracy of candidate face detection boxes, it may be necessary to delete one or more candidate face detection boxes. For example, the candidate face detection box set retained in the (i-1)th round may include detection boxes that do not contain faces (equivalent to facial features) in the target image. In such cases, at least some candidate face detection boxes in the candidate face detection box set retained in the (i-1)th round can be deleted based on, but is not limited to, the confidence set updated in the i-th round.

[0044] Optionally, in the image detection method provided in the embodiments of this application, one or more rounds of iterative processing are performed on the N candidate face detection boxes output by the target neural network model until the set of retained candidate face detection boxes is empty. It can be understood that the iterative processing ends when the set of candidate face detection boxes does not contain any candidate face detection boxes.

[0045] In an exemplary embodiment, N candidate face detection boxes can be iteratively processed until the set of retained candidate face detection boxes is empty, including, but not limited to, the following steps: Performing the i-th round of processing: extracting the corresponding candidate face detection boxes from the set of candidate face detection boxes retained in the (i-1)-th round to obtain the set of candidate face detection boxes to be determined in the i-th round, wherein, in the confidence set updated in the (i-1)-th round, the confidence of the candidate face detection boxes extracted in the i-th round is the highest; determining each candidate face detection box in the set of candidate face detection boxes retained in the (i-1)-th round other than the candidate face detection boxes extracted in the i-th round. The intersection-union ratios (IU / U) between each candidate face detection box and the candidate face detection boxes extracted in the i-th round are used to obtain the IU / U set. The IU / U set is then used to update the confidence scores in the (i-1)-th round updated confidence score set, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, to obtain the IU-th round updated confidence score set. In the IU-th round updated confidence score set, confidence scores less than a preset threshold are determined, and candidate face detection boxes with confidence scores less than the preset threshold are deleted from the IU-th round candidate face detection box set to be determined, to obtain the IU-th round retained candidate face detection box set.

[0046] Optionally, in the image detection method provided in this application embodiment, there may be confidence scores less than a preset threshold in the confidence score set updated in the i-th round. In such cases, it can be indicated that there may be unreliable candidate face detection boxes in the candidate face detection box set to be determined in the i-th round. For example, the candidate face detection box does not frame the face in the target image, or the candidate face detection box only frames part of the facial features (such as eyes, mouth, etc.) in the target image. In such cases, it can be indicated that these candidate face detection boxes may be redundant or erroneous. These potentially redundant or erroneous candidate face detection boxes are deleted from the candidate face detection box set to be determined in the i-th round, thereby achieving the screening of candidate face detection boxes and improving the accuracy of candidate face detection boxes.

[0047] Optionally, in the image detection method provided in the embodiments of this application, there may be confidence scores greater than or equal to a preset threshold in the confidence score set updated in the i-th round. In such cases, it can be indicated that there may be credible candidate face detection boxes in the candidate face detection box set to be determined in the i-th round. In such cases, candidate face detection boxes with confidence scores greater than or equal to the preset threshold can be retained in the candidate face detection box set to be determined in the i-th round, but not limited to, to obtain the candidate face detection box set retained in the i-th round.

[0048] Optionally, in the image detection method provided in this application embodiment, the area of ​​intersection between the area of ​​each candidate face detection box (excluding the candidate face detection box extracted in the i-1th round) and the area of ​​the candidate face detection box extracted in the i-1th round is determined, and the ratio of the area of ​​the intersection between the intersection between the area of ​​each candidate face detection box (excluding the candidate face detection box extracted in the i-1th round) and the area of ​​the candidate face detection box extracted in the i-1th round is determined as the total area after taking the union of the area of ​​each candidate face detection box (excluding the candidate face detection box extracted in the i-1th round) and the area of ​​the candidate face detection box extracted in the i-1th round is determined as the intersection over union ratio.

[0049] In an exemplary embodiment, the confidence scores in the confidence score set updated in the (i-1)th round, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, can be updated using the intersection-union (IU) ratio set from the i-th round, in the following manner, to obtain the confidence score set updated in the i-th round: the candidate face detection boxes in the candidate face detection box set retained in the (i-1)th round, excluding the candidate face detection boxes extracted in the i-th round, include P candidate face detection boxes; the confidence scores in the confidence score set updated in the (i-1)th round, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, include P confidence scores, the P confidence scores including the confidence scores of each candidate face detection box in the P candidate face detection boxes updated in the (i-1)th round; and the intersection-union (IU) ratio set from the i-th round includes P intersection-union ratios, the P intersection-union ratios being the confidence scores of the candidate face detection boxes extracted in the i-th round. In the case of the intersection-union ratios (IU / U) between the candidate face detection boxes and the P candidate face detection boxes, the following steps are performed to update the j-th confidence score among the P confidence scores using the j-th IU / U. Here, the j-th IU / U is the IU / U is the intersection-union ratio between the candidate face detection box extracted in the i-th round and the j-th candidate face detection box among the P candidate face detection boxes, and the j-th confidence score is the confidence score of the j-th candidate face detection box updated in the (i-1)-th round. P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: The update factor of the j-th confidence score is determined based on the j-th IU / U; the confidence score of the j-th candidate face detection box updated in the i-th round is determined based on the update factor of the j-th confidence score, the j-th confidence score, and the j-th IU / U.

[0050] Optionally, in the image detection method provided in this application embodiment, in order to reduce the impact of the preset confidence threshold on the image detection result, for example: if the preset threshold is set too high, some detection boxes in the same face detection box may not be suppressed, resulting in false detection; if the preset threshold is set too low, if two faces in the image are adjacent, and the confidence of the face detection box of the adjacent face is lower than that of the face detection box of the other face, then the face detection box with the lower confidence may be suppressed, resulting in missed face detection. The confidence of the candidate face detection box can be updated by a determined update factor, but is not limited to, thereby reducing the confidence of the candidate detection box, reducing the impact of the preset confidence threshold on the image detection result, and improving the performance of the model.

[0051] In one exemplary embodiment, the method further includes: the update factor of the j-th confidence level is negatively correlated with the j-th cross-union ratio, wherein the larger the j-th cross-union ratio, the smaller the update factor of the j-th confidence level.

[0052] Optionally, in the image detection method provided in this application embodiment, the update factor of the j-th confidence level can be determined by, but is not limited to, the following formula:

[0053] μ j =1-lg(IoU) j +1)

[0054] Among them, IoU j Let μ be the j-th intersection-union ratio. j is the update factor for the j-th confidence level.

[0055] Optionally, in the image detection method provided in this application embodiment, the confidence of the candidate face detection box can be updated by a determined update factor, but is not limited to, thereby reducing the confidence of the candidate detection box and effectively reducing the impact of the preset threshold of confidence on the detection accuracy, thus improving the accuracy of the model.

[0056] To better explain the technical effect of the preset threshold for reducing confidence on the detection accuracy in the image detection method of this application embodiments, the following explanation and description are provided in conjunction with optional embodiments, which may be applied to, but are not limited to, the embodiments of this application.

[0057] The explanation can be based on, but is not limited to, the MTCNN model used for object recognition. The training dataset contains unobstructed faces and faces wearing masks, and some images contain a large number of small faces. The dataset contains 15,000 images, divided into training, test, and validation sets in an 8:1:1 ratio, with 12,000 images in the training set, and 1,500 images each in the test and validation sets.

[0058] Using the aforementioned training dataset, we compared the impact of the preset confidence threshold on the post-processing algorithm of the improved MTCNN model as a candidate face detection box (equivalent to a candidate face detection box), and the impact of the preset confidence threshold on the post-processing algorithm of the original MTCNN model. Table 1 shows the accuracy of the original MTCNN at different thresholds T, and Table 2 shows the accuracy of the improved MTCNN at different thresholds σ.

[0059] Table 1

[0060] T 0.5 0.6 0.7 0.8 P-Net / % 86.3 86.7 87.2 86.1 R-Net / % 90.1 90.8 90.2 89.2 O-Net / % 93.3 94.7 93.2 92.2

[0061] Table 2

[0062] σ 0.002 0.003 0.004 0.005 P-Net / % 88.2 88.8 88.3 87.8 R-Net / % 92.9 93.1 93.3 93.0 O-Net / % 95.7 95.9 95.8 95.6

[0063] It should be noted that the data in Tables 1 and 2 are the average accuracy of each sub-model after convergence, calculated after 200, 140, and 100 iterations in P-Net, R-Net, and O-Net respectively, using the original MTCNN and the improved MTCNN. The data in R-Net and O-Net are obtained under the optimal threshold in the preceding sub-networks.

[0064] As shown in Tables 1 and 2, the optimal thresholds for the original MTCNN in P-Net, R-Net, and O-Net are 0.7, 0.6, and 0.6, respectively, with corresponding accuracies of 87.2%, 90.8%, and 94.7%. The optimal thresholds for the improved MTCNN in P-Net, R-Net, and O-Net are 0.003, 0.004, and 0.003, with corresponding accuracies of 88.8%, 93.3%, and 95.9%. The overall accuracy of the model applying the NMS algorithm in P-Net, R-Net, and O-Net is lower than that of the model applying the improved post-processing algorithm. The improved post-processing algorithm shows a certain improvement in detection accuracy compared to the NMS algorithm, enabling the model to maintain an accuracy of over 95% in the final output detection boxes in O-Net. The accuracy of the MTCNN model applying the original NMS algorithm varies significantly under different thresholds, indicating that the NMS algorithm is greatly affected by the threshold T. However, in the MT-NMS algorithm, the threshold σ has a much lower impact on algorithm performance than the threshold T in the NMS algorithm. When the threshold T > 0.7, the accuracy of the NMS algorithm drops significantly. The reason for this drop is that the threshold of the NMS algorithm is set too high. Although it can reduce the probability of missed face detection, some false positive face boxes that should have been filtered out by the threshold in adjacent detection boxes are not filtered out, and a large number of false positive boxes are retained. The increase of these positive samples is far greater than the number of false positives, thus causing the accuracy of the algorithm to drop.

[0065] In an exemplary embodiment, the confidence of the j-th candidate face detection box in the i-th round of updates can be determined, but is not limited to, according to the update factor of the j-th confidence, the j-th confidence, and the j-th intersection-union ratio, in the following manner: obtaining a first product of the update factor of the j-th confidence and the j-th confidence, and obtaining a first index value with the first product as the first base and a preset weight as the first exponent; obtaining a second index value with the j-th intersection-union ratio as the second base and the difference between the target value and the preset weight as the second exponent; and determining the confidence of the j-th candidate face detection box in the i-th round of updates as the product of the first index value and the second index value.

[0066] Optionally, in the image detection method provided in this application embodiment, the j-th confidence level can be updated using, but is not limited to, the following formula:

[0067] ci，j =(μj×c i-1，j ) α ×IoU j 1-α

[0068] Among them, IoU j Let μj be the crossover union ratio (CUB) of the j-th node, μj be the update factor for the j-th confidence level, and c be the crossover union ratio (CUB). i-1，j For the j-th confidence level, c i，j Let α be the confidence level of the j-th candidate face detection box in the i-th round of update, and let α be the preset weight corresponding to the intersection-union ratio (IoU) of the j-th confidence level and the j-th IoU.

[0069] Optionally, in the image detection method provided in the embodiments of this application, the preset weight α corresponding to the j-th confidence level and the j-th intersection-union ratio can be preset. For example, the value of α can be 0.7, 0.6, or 0.5, etc. This application does not limit this.

[0070] To better understand the image detection method in the embodiments of this application, the image detection method in the embodiments of this application will be explained and described below in conjunction with optional embodiments, which may be applied to the embodiments of this application but are not limited to them.

[0071] Figure 5 This is a schematic diagram of an image detection method according to an embodiment of this application, such as... Figure 5 As shown, in the case of performing the i-th round of processing, the confidence scores in the confidence score set updated in the (i-1)-th round are first sorted by size. After sorting, the candidate face detection box m with the highest confidence score is selected. s (Equivalent to the candidate face detection boxes extracted in the i-th round) are placed in the final set of candidate face detection boxes L (equivalent to the set of candidate face detection boxes retained in the i-th round).

[0072] Determine the candidate face detection box set M (equivalent to the candidate face detection box set retained in the (i-1)th round) excluding candidate face detection boxes m. s Each candidate face detection bounding box outside (e.g., candidate face detection m) j ) and candidate face detection bounding box m s Intersection over Union (IoU) j The update factor μ is calculated by crossover and union ratio. j (The update factor for the j-th candidate face detection box), through μ j Update the confidence of the j-th candidate face detection box in the i-th round of updates, thus obtaining the confidence c of the j-th candidate face detection box in the i-th round of updates. i，j c i，j =μ j ×c i-1，j , where ci-1，j Let be the confidence score of the j-th candidate face detection box in the (i-1)th round.

[0073] Then determine the confidence level c of the j-th candidate face detection box in the i-th round of updates. i，j Whether it is less than a preset threshold σ, the confidence c of the j-th candidate face detection box updated in the i-th round. i，j If the value is less than a preset threshold σ, the j-th candidate face detection box is deleted, and the confidence c of the j-th candidate face detection box is updated in the i-th round. i，j If the value is greater than or equal to the preset threshold σ, the j-th candidate face detection box will be retained.

[0074] continuously remove candidate face detection boxes m from set M (equivalent to the set of candidate face detection boxes retained in the (i-1)th round). s Select candidate face detection boxes from the candidate face detection boxes outside the original candidate face detection boxes, and repeat the above process until the number of candidate face detection boxes in set M (equivalent to the set of candidate face detection boxes retained in the (i-1)th round) is 0, and obtain the set of candidate face detection boxes retained in the i-th round.

[0075] In an exemplary embodiment, the intersection-union ratio (IU / R) between each candidate face detection box (excluding the candidate face detection boxes extracted in the i-1th round) retained in the candidate face detection box set can be determined, but is not limited to, by the following method: If the candidate face detection boxes (excluding the candidate face detection boxes extracted in the i-1th round) in the candidate face detection box set retained in the i-1th round include P candidate face detection boxes, the j-th IU / R between the j-th candidate face detection box in the P candidate face detection boxes and the candidate face detection boxes extracted in the i-th round is determined by performing the following steps, where P... Let j be a positive integer greater than or equal to 1, and let j be a positive integer greater than or equal to 1 and less than or equal to P. Input the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into the target recognition model to obtain the j-th intersection-union ratio. The target recognition model is a model obtained by training the initial recognition model to be trained using the sample face detection box set and the actual intersection-union ratio between each sample face detection box in the sample face detection box set and the sample face detection box with the highest confidence in the sample face detection box set. Each sample face detection box in the sample face detection box set has a corresponding confidence level.

[0076] Optionally, in the image detection method provided in the embodiments of this application, the target recognition model may be deployed at the output end of the target neural network model, but is not limited to. For example, taking the target neural network model as the three sub-networks included in the MTCNN model, namely the Proposal Network (P-Net), Refine Network (R-Net), and Output Network (O-Net), the target recognition model may be deployed after the pooling layer in the P-Net network, after the fully connected layer in the R-Net network, and after the fully connected layer in the O-Net network.

[0077] Optionally, in the image detection method provided in this application embodiment, the intersection-union ratio (IU / R) between the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round can be determined using a target recognition model, but is not limited to this method. Figure 6 This is a schematic diagram of an optional target recognition model according to an embodiment of this application, such as... Figure 6 As shown, taking the R-NET network in the MTCNN model as an example, but not limited to the target recognition model, an IoU output branch (equivalent to the target recognition model) is added to the NMS (non-maximum suppression) algorithm at the end of the R-NET network. Through the output of the fully connected layer, the IoU output branch predicts the IoU value (equivalent to the intersection-union ratio) between the candidate face bounding boxes (equivalent to the candidate face bounding boxes other than those extracted in the i-1th round) and the ground truth face bounding boxes (equivalent to the candidate face bounding boxes extracted in the i-1th round) and the ground truth face bounding boxes (equivalent to the candidate face bounding boxes extracted in the i-1th round). The product of the predicted IoU value (equivalent to the predicted IoU score) and the classification probability score output by the network itself (equivalent to the confidence score before the update) is used as the updated confidence score.

[0078] Compared to related technologies, where the network directly uses the classification probability score in the output branch as the model's confidence score, the classification probability score often cannot fully represent the localization accuracy in object detection. However, the image detection method in this application uses the IoU value as a factor in a new confidence formula, which makes object localization more accurate to a certain extent and improves the accuracy of candidate face detection boxes.

[0079] In an exemplary embodiment, the initial recognition model can be trained to obtain the target recognition model in the following manner, but not limited to: performing the k-th round of training on the initial recognition model to be trained through the following steps, where k is a positive integer greater than or equal to 2: inputting the k-th set of sample face detection boxes used in the k-th round into the initial recognition model obtained from the (k-1)-th round of training to obtain the k-th set of predicted crossover ratios, wherein the set of sample face detection boxes includes the k-th set of sample face detection boxes, and the k-th set of sample face detection boxes includes Q k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k A positive integer greater than or equal to 2; the loss function for the k-th round is determined based on the actual cross-union ratio (CUN) of the k-th group used in the k-th round and the predicted CUN of the k-th group, wherein the actual CUN of the k-th group includes the Q... k -1 Sample face detection box and Q respectively k The actual crossover ratio between the face detection boxes with the highest confidence in each sample face detection box; if the loss function in the k-th round satisfies the preset convergence condition, the training of the initial recognition model to be trained ends, and the initial recognition model obtained in the (k-1)-th round of training is determined as the target recognition model; if the loss function in the k-th round does not satisfy the convergence condition, the parameters in the initial recognition model obtained in the (k-1)-th round of training are adjusted to obtain the initial recognition model obtained in the k-th round of training.

[0080] Optionally, in the image detection method provided in this application embodiment, if the loss function in the k-th round satisfies the preset convergence condition, it indicates that the training of the initial recognition model is complete. In this case, the training of the initial recognition model to be trained ends. If the loss function in the k-th round does not satisfy the convergence condition, the parameters in the initial recognition model obtained in the (k-1)-th round of training are adjusted, and then the initial model with adjusted parameters is trained again until the loss function satisfies the convergence condition, and the training of the initial recognition model ends, thus obtaining the target recognition model. Through the image detection method in this application embodiment, although a new IOU output branch (equivalent to the target recognition model) is added to the network, the total loss is not affected by this IOU output branch. Furthermore, after adding the IOU output branch, the time complexity of the algorithm does not increase, and the computational load does not increase significantly, thus improving the utilization efficiency of computer resources.

[0081] In an exemplary embodiment, the loss function for the k-th round can be determined, but is not limited to, by: obtaining a set of cross-entropy loss values ​​based on a first set of actual cross-union ratios and a second set of actual cross-union ratios, wherein the first set of actual cross-union ratios includes W actual cross-union ratios, the second set of actual cross-union ratios includes W actual cross-union ratios, the set of cross-entropy loss values ​​includes W cross-entropy loss values, and the i-th actual cross-union ratio in the first set of actual cross-union ratios is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence in the second set of face detection boxes is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual cross-entropy ratio (COP) between the face detection boxes with the highest confidence among the sample face detection boxes is given by the W cross-entropy loss values. The i-th cross-entropy loss value is obtained by performing cross-entropy loss calculation on the i-th actual cross-entropy ratio in the first group and the i-th actual cross-entropy ratio in the second group. W equals Q. k -1, where i is greater than or equal to 1 and less than or equal to Q. k -1 is a positive integer; the loss function of the k-th round is determined as the average of the W actual cross-union ratios.

[0082] Optionally, in the image detection method provided in this application embodiment, the loss function for the k-th round can be determined by, but is not limited to, the following formula:

[0083]

[0084] Where W equals Q k-1,LIoU k Let IoU be the loss function for the k-th round. i For the Q k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence among the sample face detection boxes. For the Q k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual intersection-union ratio (CUC) between the face detection boxes with the highest confidence among the sample face detection boxes is denoted by CE, where CE represents the cross-entropy loss operation, and i is greater than or equal to 1 and less than or equal to Q. k -1.

[0085] Optionally, in the image detection method provided in this application embodiment, it is possible, but not limited to, determining that the loss function of the k-th round satisfies the preset convergence condition when the loss function value of the k-th round is less than or equal to the preset loss function threshold; it is possible, but not limited to, determining that the loss function of the k-th round does not satisfy the preset convergence condition when the loss function value of the k-th round is greater than the preset loss function threshold.

[0086] In the technical solution provided in step S103 above, at least M candidate face detection boxes selected from N candidate face detection boxes can be determined as the results detected in the target image, and the M candidate face detection boxes are displayed in the target image. After completing M rounds of iterative processing, the final reliable candidate face detection boxes include at least the candidate face detection boxes with the highest confidence extracted in each round of iterative processing. In this way, the accuracy of face detection in the target image is improved.

[0087] Optionally, in the image detection method provided in this application embodiment, after completing M rounds of iterative processing, the final reliable candidate face detection box may also include candidate face detection boxes corresponding to a confidence level greater than or equal to a preset threshold.

[0088] To better explain the technical effect of improving the accuracy of image detection in the image detection method of this application embodiments, the following explanation and description are provided in conjunction with optional embodiments, which may be applied to, but are not limited to, the embodiments of this application.

[0089] The image detection method in this application embodiment can be applied, but is not limited to, to security inspection systems, facial recognition verification, mobile phone unlocking, and other scenarios requiring face detection. This method avoids missed face detection when faces in the target image are occluded. Figure 7This is a schematic diagram illustrating face detection when the face is occluded, according to an embodiment of this application. Figure 7 As shown, the face of person B in image A is covered by an occlusion (e.g., a mask). In related technologies, person B is often missed, and even with model optimization by adjusting the threshold, it is difficult to achieve good detection results. However, with the image detection method in this application embodiment, although person B's face is partially covered by a mask, it is still possible to detect person B's face and display the face detection box of person B's face in image B, effectively reducing the missed detection rate and improving the detection performance of occluded faces.

[0090] The image detection method in this application not only improves the detection performance of occluded faces, but also improves the detection performance of smaller faces. Figure 8 This is a schematic diagram of a small face detection according to an embodiment of this application, such as... Figure 8 As shown, image C contains face 1, face 2, and face 3. Face 1 and face 2 are adjacent, and the size of face 1 is smaller than that of face 2. Related technologies can only detect face 2 and face 3, but not face 1. However, the image detection method described in this embodiment can detect face 1, face 2, and face 3, improving the detection efficiency for adjacent faces in an image.

[0091] To further verify the performance of the improved post-processing algorithm, tests were conducted on the publicly available face test set FDDB. Figure 9 This is a schematic diagram of the ROC curve on an FDDB test set according to an embodiment of this application, as shown below. Figure 9 As shown, the ROC curve's horizontal axis represents FPR (False Positive Rate), and the vertical axis represents TPR (True Positive Rate). At each confidence threshold, there exists a set of FPR and TPR values, representing the false positive rate and recall rate of the model at different confidence thresholds. In the ROC curve, the closer the curve is to the upper left, the higher the recall rate and the lower the false positive rate, indicating better model performance. Figure 9 It can be seen that the ROC curve of the optimized model is closer to the upper left than the ROC curve of the unoptimized model. Therefore, the MTCNN model with MT-NMS and the new confidence formula can effectively reduce the false detection rate and improve the recall rate.

[0092] It should be noted that the acquisition, storage, use, and processing of data involved in the embodiments of this application all comply with the relevant provisions of national laws and regulations.

[0093] The image detection method provided in this application performs multiple rounds of iterative processing on the N candidate face detection boxes output by the target neural network model. During the i-th iteration, candidate face detection boxes are extracted from the set of candidate face detection boxes retained in the (i-1)-th iteration based on the updated confidence set, until the set of retained candidate face detection boxes is empty. In the previous iteration, the confidence set required for the next iteration is updated, and the candidate face detection boxes in the set of retained candidate face detection boxes from the previous iteration are filtered based on the updated confidence set. This solves the problem of low accuracy in image detection in related technologies, thereby improving the accuracy of image detection.

[0094] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0095] This application also provides an image detection apparatus. It should be noted that the image detection apparatus of this application can be used to execute the image detection method provided in this application. The image detection apparatus provided in this application is described below.

[0096] Figure 10 This is a schematic diagram of an image detection apparatus according to an embodiment of this application. Figure 10 As shown, the device includes:

[0097] The first input module 1002 is used to obtain N candidate face detection boxes and N initial confidence scores detected in the target image, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence.

[0098] The iterative processing module 1004 is used to iteratively process the N candidate face detection boxes until the set of retained candidate face detection boxes is empty. The iterative processing includes M rounds of processing, extracting a total of M candidate face detection boxes through the M rounds, where M is a positive integer greater than or equal to 1. The candidate face detection boxes extracted in the i-th round are obtained from the set of retained candidate face detection boxes in the (i-1)-th round based on the confidence set updated in the (i-1)-th round. In the i-th round, candidate face detection boxes need to be deleted from the set of retained candidate face detection boxes in the (i-1)-th round. In the case of detection boxes, the deleted candidate face detection boxes are candidate face detection boxes determined according to the confidence set updated in the i-th round. The confidence set updated in the (i-1)-th round includes the confidence of each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round. When i = 1, the confidence set updated in the (i-1)-th round includes the N initial confidences, and the candidate face detection box set retained in the (i-1)-th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M.

[0099] The processing module 1006 is used to determine the M candidate face detection boxes as the results detected in the target image, and to display the M candidate face detection boxes in the target image.

[0100] Optionally, in the image detection apparatus provided in this application embodiment, the iterative processing module is used to perform the i-th round of processing through the following units: an extraction unit, used to extract corresponding candidate face detection boxes from the candidate face detection box set retained in the (i-1)-th round, to obtain the candidate face detection box set to be determined in the i-th round, wherein, in the confidence set updated in the (i-1)-th round, the confidence of the candidate face detection boxes extracted in the i-th round is the highest; a determination unit, used to determine that each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round, excluding the candidate face detection boxes extracted in the i-th round, is respectively related to the i-th round of processing. The intersection-union ratio (CUN) between candidate face detection boxes extracted in round i is used to obtain the CUN set for round i. An update unit is used to update the confidence scores in the confidence score set updated in round i-1 (excluding the confidence scores of the candidate face detection boxes extracted in round i) using the CUN set for round i, thus obtaining the updated confidence score set for round i. A deletion unit is used to determine the confidence scores in the updated confidence score set for round i that are less than a preset threshold, and delete the candidate face detection boxes corresponding to the confidence scores less than the preset threshold from the candidate face detection box set to be determined in round i, thus obtaining the candidate face detection box set retained in round i.

[0101] Optionally, in the image detection apparatus provided in this application embodiment, the updating unit is configured to: in the candidate face detection box set retained in the (i-1)th round, the candidate face detection boxes other than those extracted in the i-th round include P candidate face detection boxes; in the confidence score set updated in the (i-1)th round, the confidence scores other than those of the candidate face detection boxes extracted in the i-th round include P confidence scores, the P confidence scores including the confidence scores of each candidate face detection box in the P candidate face detection boxes updated in the (i-1)th round; and in the intersection-union ratio (IU) set of the i-th round, the IU set includes P IU ratios, where the P IU ratios are the IU ratios between the candidate face detection boxes extracted in the i-th round and the P candidate face detection boxes, by executing... The following steps involve updating the j-th confidence score among the P confidence scores using the j-th intersection-union ratio among the P intersection-union ratios, where the j-th intersection-union ratio is the intersection-union ratio between the candidate face detection box extracted in the i-th round and the j-th candidate face detection box among the P candidate face detection boxes, and the j-th confidence score is the confidence score of the j-th candidate face detection box updated in the (i-1)-th round, where P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: determining the update factor of the j-th confidence score based on the j-th intersection-union ratio; and determining the confidence score of the j-th candidate face detection box updated in the i-th round based on the update factor of the j-th confidence score, the j-th confidence score, and the j-th intersection-union ratio.

[0102] Optionally, in the image detection apparatus provided in the embodiments of this application, the updating unit is further configured to: the updating factor of the j-th confidence level is negatively correlated with the j-th cross-union ratio, wherein the larger the j-th cross-union ratio, the smaller the updating factor of the j-th confidence level.

[0103] Optionally, in the image detection apparatus provided in this application embodiment, the updating unit is further configured to: update the j-th confidence level using the following formula: obtain the first product of the update factor of the j-th confidence level and the j-th confidence level, and obtain a first index value using the first product as the first base and a preset weight as the first exponent; obtain a second index value using the j-th intersection-union ratio as the second base and the difference between the target value and the preset weight as the second exponent; and determine the confidence level of the j-th candidate face detection box in the i-th round of update as the product of the first index value and the second index value.

[0104] Optionally, in the image detection apparatus provided in this application embodiment, the determining unit is configured to: when the candidate face detection boxes in the candidate face detection box set retained in the (i-1)th round include P candidate face detection boxes other than the candidate face detection boxes extracted in the i-th round, determine the j-th intersection-union ratio between the j-th candidate face detection box in the P candidate face detection boxes and the candidate face detection boxes extracted in the i-th round by performing the following steps, where P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P; input the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into a target recognition model to obtain the j-th intersection-union ratio, wherein the target recognition model is a model obtained by training an initial recognition model to be trained using a sample face detection box set and the actual intersection-union ratio between each sample face detection box in the sample face detection box set and the sample face detection box with the highest confidence in the sample face detection box set, and each sample face detection box in the sample face detection box set has a corresponding confidence level.

[0105] Optionally, in the image detection apparatus provided in this application embodiment, the apparatus further includes: a second input module, used to perform a k-th round of training on the initial recognition model to be trained through the following module before inputting the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into the target recognition model, wherein k is a positive integer greater than or equal to 2: a second input module, used to input the k-th group of sample face detection boxes used in the k-th round into the initial recognition model obtained in the (k-1)-th round of training to obtain the k-th group of predicted intersection-union ratios, wherein the set of sample face detection boxes includes the k-th group of sample face detection boxes, and the k-th group of sample face detection boxes includes Q k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k The first determining module is configured to determine the loss function for the k-th round based on the actual cross-union ratio of the k-th group used in the k-th round and the predicted cross-union ratio of the k-th group, wherein the actual cross-union ratio of the k-th group includes the Q... k -1 Sample face detection box and Q respectivelyk The actual crossover ratio between the face detection boxes with the highest confidence in each sample face detection box; the second determining module is used to end the training of the initial recognition model to be trained when the loss function in the k-th round meets the preset convergence condition, and determine the initial recognition model obtained in the (k-1)-th round of training as the target recognition model; when the loss function in the k-th round does not meet the convergence condition, adjust the parameters in the initial recognition model obtained in the (k-1)-th round of training to obtain the initial recognition model obtained in the k-th round of training.

[0106] Optionally, in the image detection apparatus provided in this application embodiment, the first determining module is used to obtain a set of cross-entropy loss values ​​based on a first set of actual cross-union ratios and a second set of actual cross-union ratios, wherein the first set of actual cross-union ratios includes W actual cross-union ratios, the second set of actual cross-union ratios includes W actual cross-union ratios, the set of cross-entropy loss values ​​includes W cross-entropy loss values, and the i-th actual cross-union ratio in the first set of actual cross-union ratios is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence in the second set of face detection boxes is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual cross-entropy ratio (COP) between the face detection boxes with the highest confidence among the sample face detection boxes is given by the W cross-entropy loss values. The i-th cross-entropy loss value is obtained by performing cross-entropy loss calculation on the i-th actual cross-entropy ratio in the first group and the i-th actual cross-entropy ratio in the second group. W equals Q. k -1, where i is greater than or equal to 1 and less than or equal to Q. k -1 is a positive integer; the loss function of the k-th round is determined as the average of the W actual cross-union ratios.

[0107] The image detection apparatus provided in this application performs multiple rounds of iterative processing on N candidate face detection boxes output by the target neural network model. During the i-th iteration, candidate face detection boxes are extracted from the set of candidate face detection boxes retained in the (i-1)-th iteration based on the updated confidence set, until the retained set of candidate face detection boxes is empty. In the previous iteration, the confidence set required for the next iteration is updated, and the candidate face detection boxes in the set of candidate face detection boxes retained in the previous iteration are filtered based on the updated confidence set. This solves the problem of low accuracy in image detection in related technologies, thereby improving the accuracy of image detection.

[0108] The image detection device includes a processor and a memory. The first input module 1002, the iterative processing module 1004, and the processing module 1006 are all stored in the memory as program units. The processor executes the program units stored in the memory to achieve the corresponding functions.

[0109] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve the accuracy of image detection.

[0110] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0111] This application provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the image detection method.

[0112] This application provides a processor for running a program, wherein the program executes the image detection method during runtime.

[0113] Figure 11 This is a schematic diagram of an electronic device according to an embodiment of this application, such as... Figure 11As shown, this application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring N candidate face detection boxes and N initial confidence scores detected in a target image, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence; iteratively processing the N candidate face detection boxes until the set of retained candidate face detection boxes is empty, wherein the iterative processing includes M rounds of processing, and a total of M candidate face detection boxes are extracted after the M rounds of processing, where M is a positive integer greater than or equal to 1, and the candidate face detection boxes extracted in the i-th round are extracted from the set of candidate face detection boxes retained in the (i-1)-th round based on the confidence score set updated in the (i-1)-th round. In the i-th round of processing, if it is necessary to delete candidate face detection boxes from the candidate face detection box set retained in the (i-1)-th round, the deleted candidate face detection boxes are candidate face detection boxes determined according to the confidence set updated in the i-th round. The confidence set updated in the (i-1)-th round includes the confidence scores of each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round. When i = 1, the confidence set updated in the (i-1)-th round includes the N initial confidence scores. The candidate face detection box set retained in the (i-1)-th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M. The M candidate face detection boxes are determined as the results detected in the target image, and the M candidate face detection boxes are displayed in the target image.

[0114] Optionally, the processor may also perform the following steps when executing the program: Perform the i-th round of processing by: extracting corresponding candidate face detection boxes from the candidate face detection box set retained in the (i-1)-th round, to obtain the candidate face detection box set to be determined in the i-th round, wherein, in the confidence set updated in the (i-1)-th round, the confidence of the candidate face detection boxes extracted in the i-th round is the highest; determining that each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round, excluding the candidate face detection boxes extracted in the i-th round, is respectively compared with the candidate face detection boxes extracted in the i-th round. The intersection-union ratio (CUN) of candidate face detection boxes is used to obtain the CUN set for the i-th round. The confidence scores of the confidence scores in the (i-1)-th round updated confidence score set, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, are updated using the CUN set for the i-th round to obtain the updated confidence score set for the i-th round. In the updated confidence score set for the i-th round, confidence scores less than a preset threshold are determined, and candidate face detection boxes with confidence scores less than the preset threshold are deleted from the candidate face detection box set to be determined in the i-th round, thus obtaining the candidate face detection box set retained in the i-th round.

[0115] Optionally, the processor may further implement the following steps when executing the program: In the candidate face detection box set retained in the (i-1)th round, the candidate face detection boxes other than those extracted in the i-th round include P candidate face detection boxes; in the confidence score set updated in the (i-1)th round, the confidence scores other than those of the candidate face detection boxes extracted in the i-th round include P confidence scores, where the P confidence scores include the confidence scores of each candidate face detection box in the P candidate face detection boxes updated in the (i-1)th round; and in the intersection-union ratio (IU) set for the i-th round, the IU set includes P IU ratios, where the P IU ratios are the IU ratios of the candidate face detection boxes extracted in the i-th round with the P candidate face detection boxes respectively. Then, by executing the following steps... The j-th confidence score among the P confidence scores is updated using the j-th intersection-union ratio among the P intersection-union ratios, where the j-th intersection-union ratio is the intersection-union ratio between the candidate face detection box extracted in the i-th round and the j-th candidate face detection box among the P candidate face detection boxes, and the j-th confidence score is the confidence score of the j-th candidate face detection box updated in the (i-1)-th round, where P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P. The update factor of the j-th confidence score is determined based on the j-th intersection-union ratio. The confidence score of the j-th candidate face detection box updated in the i-th round is determined based on the update factor of the j-th confidence score, the j-th confidence score, and the j-th intersection-union ratio.

[0116] Optionally, when the processor executes the program, it may also implement the following steps: the update factor of the j-th confidence level is negatively correlated with the j-th cross-union ratio, wherein the larger the j-th cross-union ratio, the smaller the update factor of the j-th confidence level.

[0117] Optionally, the processor may also perform the following steps when executing the program: obtaining the update factor of the j-th confidence level and the first product of the j-th confidence level, and obtaining a first index value with the first product as the first base and the preset weight as the first exponent; obtaining a second index value with the j-th intersection-union ratio as the second base and the difference between the target value and the preset weight as the second exponent; and determining the confidence level of the j-th candidate face detection box in the i-th round of update as the product of the first index value and the second index value.

[0118] Optionally, the processor may also implement the following steps when executing the program: If the candidate face detection boxes in the (i-1)th round retained include P candidate face detection boxes other than those extracted in the i-th round, determine the j-th intersection-union ratio (IU / R) between the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round by performing the following steps: where P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P; input the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into the target recognition model to obtain the j-th IU / R, where the target recognition model is a model trained using a sample face detection box set and the actual IU / R between each sample face detection box in the sample face detection box set and the sample face detection box with the highest confidence in the sample face detection box set, and each sample face detection box in the sample face detection box set has a corresponding confidence level.

[0119] Optionally, the processor may also implement the following steps when executing the program: Perform the k-th round of training on the initial recognition model to be trained using the following steps, where k is a positive integer greater than or equal to 2: Input the k-th group of sample face detection boxes used in the k-th round into the initial recognition model obtained from the (k-1)-th round of training to obtain the k-th group of predicted crossover ratios, wherein the set of sample face detection boxes includes the k-th group of sample face detection boxes, and the k-th group of sample face detection boxes includes Q... k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k A positive integer greater than or equal to 2; the loss function for the k-th round is determined based on the actual cross-union ratio (CUN) of the k-th group used in the k-th round and the predicted CUN of the k-th group, wherein the actual CUN of the k-th group includes the Q... k -1 Sample face detection box and Q respectively kThe actual crossover ratio between the face detection boxes with the highest confidence in each sample face detection box; if the loss function in the k-th round satisfies the preset convergence condition, the training of the initial recognition model to be trained ends, and the initial recognition model obtained in the (k-1)-th round of training is determined as the target recognition model; if the loss function in the k-th round does not satisfy the convergence condition, the parameters in the initial recognition model obtained in the (k-1)-th round of training are adjusted to obtain the initial recognition model obtained in the k-th round of training.

[0120] Optionally, the processor may further implement the following steps when executing the program: obtaining a set of cross-entropy loss values ​​based on a first set of actual cross-union ratios and a second set of actual cross-union ratios, wherein the first set of actual cross-union ratios includes W actual cross-union ratios, the second set of actual cross-union ratios includes W actual cross-union ratios, the set of cross-entropy loss values ​​includes W cross-entropy loss values, and the i-th actual cross-union ratio in the first set of actual cross-union ratios is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence in the second set of face detection boxes is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual cross-entropy ratio (COP) between the face detection boxes with the highest confidence among the sample face detection boxes is given by the W cross-entropy loss values. The i-th cross-entropy loss value is obtained by performing cross-entropy loss calculation on the i-th actual cross-entropy ratio in the first group and the i-th actual cross-entropy ratio in the second group. W equals Q. k -1, where i is greater than or equal to 1 and less than or equal to Q. k -1 is a positive integer; the loss function of the k-th round is determined as the average of the W actual cross-union ratios.

[0121] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0122] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program with the following initialization steps: acquiring N candidate face detection boxes detected in a target image and N initial confidence scores, where N is a positive integer greater than or equal to 1, and the N candidate face detection boxes and the N initial confidence scores have a one-to-one correspondence; iteratively processing the N candidate face detection boxes until the set of retained candidate face detection boxes is empty, wherein the iterative processing includes M rounds of processing, and a total of M candidate face detection boxes are extracted after the M rounds of processing, where M is a positive integer greater than or equal to 1, and the candidate face detection boxes extracted in the i-th round are extracted from the set of candidate face detection boxes retained in the (i-1)-th round based on the confidence score set updated in the (i-1)-th round, and in the i-th round of processing... When deleting candidate face detection boxes from the set of candidate face detection boxes retained in the (i-1)th round, the deleted candidate face detection boxes are those determined based on the confidence set updated in the (i-1)th round. The confidence set updated in the (i-1)th round includes the confidence scores of each candidate face detection box in the set of candidate face detection boxes retained in the (i-1)th round. When i = 1, the confidence set updated in the (i-1)th round includes the N initial confidence scores. The set of candidate face detection boxes retained in the (i-1)th round includes the N candidate face detection boxes, where i is a positive integer greater than or equal to 1 and less than or equal to M. The M candidate face detection boxes are determined as the results detected in the target image, and the M candidate face detection boxes are displayed in the target image.

[0123] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program with the following method steps: performing the i-th round of processing through the following steps: extracting the corresponding candidate face detection boxes from the candidate face detection box set retained in the (i-1)-th round to obtain the candidate face detection box set to be determined in the i-th round, wherein, in the confidence set updated in the (i-1)-th round, the confidence of the candidate face detection boxes extracted in the i-th round is the highest; determining each candidate face detection box in the candidate face detection box set retained in the (i-1)-th round other than the candidate face detection boxes extracted in the i-th round. The intersection-union ratios (IU / U) between the candidate face detection boxes extracted in the i-th round and the i-th round are used to obtain the i-th round IU / U set. The confidence scores in the (i-1)-th round updated confidence score set, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, are updated using the i-th round IU / U set to obtain the i-th round updated confidence score set. In the i-th round updated confidence score set, the confidence scores that are less than a preset threshold are determined, and the candidate face detection boxes with confidence scores less than the preset threshold are deleted from the i-th round candidate face detection box set to be determined, thus obtaining the i-th round retained candidate face detection box set.

[0124] Optionally, when the above-described computer program product is executed on a data processing device, it is suitable to execute a program with the following initialization steps: The candidate face detection boxes in the set of candidate face detection boxes retained in the (i-1)th round, excluding the candidate face detection boxes extracted in the i-th round, include P candidate face detection boxes; the confidence scores in the set of confidence scores updated in the (i-1)th round, excluding the confidence scores of the candidate face detection boxes extracted in the i-th round, include P confidence scores, where the P confidence scores include the confidence scores of each candidate face detection box in the P candidate face detection boxes updated in the (i-1)th round; the cross-union ratio (CUN) set in the i-th round includes P CUNs, where the P CUNs are the cross-union ratios between the candidate face detection boxes extracted in the i-th round and the P candidate face detection boxes respectively. The following steps are performed to update the j-th confidence score among the P confidence scores using the j-th intersection-union ratio among the P intersection-union ratios, wherein the j-th intersection-union ratio is the intersection-union ratio between the candidate face detection box extracted in the i-th round and the j-th candidate face detection box among the P candidate face detection boxes, and the j-th confidence score is the confidence score of the j-th candidate face detection box updated in the (i-1)-th round, where P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: determining the update factor of the j-th confidence score based on the j-th intersection-union ratio; and determining the confidence score of the j-th candidate face detection box updated in the i-th round based on the update factor of the j-th confidence score, the j-th confidence score, and the j-th intersection-union ratio.

[0125] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program with the following method steps: the update factor of the j-th confidence level is negatively correlated with the j-th cross-union ratio, wherein the larger the j-th cross-union ratio, the smaller the update factor of the j-th confidence level.

[0126] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program with the following method steps: obtaining the update factor of the j-th confidence level and the first product of the j-th confidence level, and obtaining a first index value with the first product as the first base and a preset weight as the first exponent; obtaining a second index value with the j-th intersection-union ratio as the second base and the difference between the target value and the preset weight as the second exponent; and determining the confidence level of the j-th candidate face detection box in the i-th round of update as the product of the first index value and the second index value.

[0127] Optionally, when the above-described computer program product is executed on a data processing device, it is suitable to execute a program that initializes with the following method steps: If the set of candidate face detection boxes retained in the (i-1)th round contains P candidate face detection boxes other than those extracted in the i-th round, the program determines the j-th intersection-union ratio (IUU) between the j-th candidate face detection box in the P candidate face detection boxes and the candidate face detection boxes extracted in the i-th round by performing the following steps, where P is a positive integer greater than or equal to 1, and j is a integer greater than or equal to 1 and... A positive integer less than or equal to P; input the j-th candidate face detection box and the candidate face detection boxes extracted in the i-th round into the target recognition model to obtain the j-th intersection-union ratio, wherein the target recognition model is a model obtained by training the initial recognition model to be trained using the sample face detection box set and the actual intersection-union ratio between each sample face detection box in the sample face detection box set and the sample face detection box with the highest confidence in the sample face detection box set, and each sample face detection box in the sample face detection box set has a corresponding confidence.

[0128] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program having the following method steps: performing the k-th round of training on the initial recognition model to be trained through the following steps, where k is a positive integer greater than or equal to 2: inputting the k-th group of sample face detection boxes used in the k-th round into the initial recognition model obtained from the (k-1)-th round of training, and obtaining the k-th group of predicted crossover ratios, wherein the set of sample face detection boxes includes the k-th group of sample face detection boxes, and the k-th group of sample face detection boxes includes Q k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k A positive integer greater than or equal to 2; the loss function for the k-th round is determined based on the actual cross-union ratio (CUN) of the k-th group used in the k-th round and the predicted CUN of the k-th group, wherein the actual CUN of the k-th group includes the Q... k -1 Sample face detection box and Q respectively kThe actual crossover ratio between the face detection boxes with the highest confidence in each sample face detection box; if the loss function in the k-th round satisfies the preset convergence condition, the training of the initial recognition model to be trained ends, and the initial recognition model obtained in the (k-1)-th round of training is determined as the target recognition model; if the loss function in the k-th round does not satisfy the convergence condition, the parameters in the initial recognition model obtained in the (k-1)-th round of training are adjusted to obtain the initial recognition model obtained in the k-th round of training.

[0129] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program having the following method steps: obtaining a set of cross-entropy loss values ​​based on a first set of actual cross-union ratios and a second set of actual cross-union ratios, wherein the first set of actual cross-union ratios includes W actual cross-union ratios, the second set of actual cross-union ratios includes W actual cross-union ratios, the set of cross-entropy loss values ​​includes W cross-entropy loss values, and the i-th actual cross-union ratio in the first set of actual cross-union ratios is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual crossover ratio (CLORD) between the face detection boxes with the highest confidence in the second set of face detection boxes is Q. k -1 The i-th sample face detection box in the sample face detection box and the Q k The actual cross-entropy ratio (COP) between the face detection boxes with the highest confidence among the sample face detection boxes is given by the W cross-entropy loss values. The i-th cross-entropy loss value is obtained by performing cross-entropy loss calculation on the i-th actual cross-entropy ratio in the first group and the i-th actual cross-entropy ratio in the second group. W equals Q. k -1, where i is greater than or equal to 1 and less than or equal to Q. k -1 is a positive integer; the loss function of the k-th round is determined as the average of the W actual cross-union ratios.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image detection method characterized by, The method comprises the following steps: obtaining N candidate face detection boxes and N initial confidences detected in a target image, N being a positive integer greater than or equal to 1, the N candidate face detection boxes and the N initial confidences having a one-to-one correspondence relationship; performing iterative processing on the N candidate face detection boxes until a set of retained candidate face detection boxes is empty, wherein the iterative processing comprises M rounds of processing, M being a positive integer greater than or equal to 1, M candidate face detection boxes being extracted through the M rounds of processing, the candidate face detection boxes extracted in the i th round being extracted from a set of retained candidate face detection boxes in the i th-1 round according to an updated confidence set in the i th-1 round, in the case where it is necessary to delete candidate face detection boxes from the set of retained candidate face detection boxes in the i th-1 round in the i th round of processing, the deleted candidate face detection boxes being determined according to an updated confidence set in the i th round, the updated confidence set in the i th-1 round comprising the confidence of each candidate face detection box in the set of retained candidate face detection boxes in the i th-1 round in the updated confidence in the i th-1 round, when i = 1, the updated confidence set in the i th-1 round comprising the N initial confidences, the set of retained candidate face detection boxes in the i th-1 round comprising the N candidate face detection boxes, i being a positive integer greater than or equal to 1 and less than or equal to M; determining the M candidate face detection boxes as results of detection in the target image, and displaying the M candidate face detection boxes in the target image; wherein the iterative processing on the N candidate face detection boxes until the set of retained candidate face detection boxes is empty comprises performing the i th round of processing by the following steps: extracting corresponding candidate face detection boxes from the set of retained candidate face detection boxes in the i th-1 round to obtain a set of candidate face detection boxes to be determined in the i th round, wherein the confidence of the candidate face detection boxes extracted in the i th round is the highest in the updated confidence set in the i th-1 round; determining the intersection over union between each candidate face detection box in the set of retained candidate face detection boxes in the i th-1 round except the candidate face detection boxes extracted in the i th round and the candidate face detection boxes extracted in the i th round to obtain a set of intersection over union in the i th round; updating the confidences in the updated confidence set in the i th-1 round except the confidence of the candidate face detection boxes extracted in the i th round using the set of intersection over union in the i th round to obtain an updated confidence set in the i th round; determining the confidences less than a preset threshold in the updated confidence set in the i th round, and deleting the candidate face detection boxes corresponding to the confidences less than the preset threshold in the set of candidate face detection boxes to be determined in the i th round to obtain a set of retained candidate face detection boxes in the i th round. The updating of the confidence other than the confidence of the candidate face detection frame extracted in the i-th round in the confidence set updated in the i-1-th round by using the i-th round IoU set comprises: the candidate face detection frame other than the candidate face detection frame extracted in the i-th round in the candidate face detection frame set reserved in the i-1-th round comprises P candidate face detection frames, the confidence other than the confidence of the candidate face detection frame extracted in the i-th round in the confidence set updated in the i-1-th round comprises P confidences, the P confidences comprise the confidence of each candidate face detection frame in the P candidate face detection frames in the confidence set updated in the i-1-th round, and the i-th round IoU set comprises P IoUs, which are the IoUs between the candidate face detection frame extracted in the i-th round and the P candidate face detection frames; by performing the following steps, the j-th confidence in the P confidences is updated by using the j-th IoU in the P IoUs, wherein the j-th IoU is the IoU between the candidate face detection frame extracted in the i-th round and the j-th candidate face detection frame in the P candidate face detection frames, and the j-th confidence is the confidence of the j-th candidate face detection frame in the confidence set updated in the i-1-th round, P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: determining an update factor of the j-th confidence according to the j-th IoU; determining the confidence of the j-th candidate face detection frame in the confidence set updated in the i-th round according to the update factor of the j-th confidence, the j-th confidence, and the j-th IoU; The update factor of the j-th confidence is negatively correlated with the j-th IoU, wherein the greater the j-th IoU is, the smaller the update factor of the j-th confidence is; The updating factor of the jth confidence is determined according to the jth intersection over union, including: determining the updating factor of the jth confidence by the following formula: wherein, is the updating factor of the jth confidence, is the jth intersection over union.

2. The method of claim 1, wherein, The determination of the confidence of the j-th candidate face detection frame in the confidence set updated in the i-th round according to the update factor of the j-th confidence, the j-th confidence, and the j-th IoU comprises: obtaining a first product of the update factor of the j-th confidence and the j-th confidence, and obtaining a first index value by taking the first product as a first base and taking a preset weight as a first index; obtaining a second index value by taking the j-th IoU as a second base and taking a difference between a target value and the preset weight as a second index; determining the confidence of the j-th candidate face detection frame in the confidence set updated in the i-th round as a product of the first index value and the second index value.

3. The method of claim 1, wherein, The determination of the IoU between each candidate face detection frame other than the candidate face detection frame extracted in the i-th round in the candidate face detection frame set reserved in the i-1-th round and the candidate face detection frame extracted in the i-th round comprises: In a case where the candidate face bounding boxes in the candidate face bounding box set reserved in the i-1th round of reservation except the candidate face bounding box extracted in the i th round include P candidate face bounding boxes, P being a positive integer greater than or equal to 1, the j th intersection over union between the j th candidate face bounding box in the P candidate face bounding boxes and the candidate face bounding box extracted in the i th round is determined by the following steps, j being a positive integer greater than or equal to 1 and less than or equal to P; the j th candidate face bounding box and the candidate face bounding box extracted in the i th round are input into a target recognition model to obtain the j th intersection over union, the target recognition model being a model obtained by training an initial recognition model to be trained using a sample face bounding box set and actual intersection over unions between each sample face bounding box in the sample face bounding box set and a sample face bounding box with the highest confidence in the sample face bounding box set, each sample face bounding box in the sample face bounding box set having a corresponding confidence.

4. The method of claim 3, wherein, Before the j th candidate face bounding box and the candidate face bounding box extracted in the i th round are input into the target recognition model, the method further includes: the k th round of training is performed on the initial recognition model to be trained by the following steps, k being a positive integer greater than or equal to 2: The k-th set of sample face detection boxes used in the k-th round is input into the initial recognition model obtained from the (k-1)-th round of training to obtain the k-th predicted intersection-union ratio, wherein the set of sample face detection boxes includes the k-th set of sample face detection boxes, and the k-th set of sample face detection boxes includes Q k A sample face detection bounding box, the k-th group of predicted crossover ratios includes Q... k -1 crossover ratio, the Q k -1 Cross-Union Ratios (CUI) include the Q predicted by the initial recognition model obtained from the (k-1)th round of training. k -1 Sample face detection box and Q respectively k The crossover ratio (CLOUD) between the face detection boxes with the highest confidence among the sample face detection boxes, the Q k -1 Sample face detection frame includes the Q k The sample face detection boxes other than the sample face detection box with the highest confidence level in the sample face detection boxes, Q k It is a positive integer greater than or equal to 2; According to the kth set of actual intersection-over-union and the kth set of predicted intersection-over-union used in the kth round, a loss function of the kth round is determined, wherein the kth set of actual intersection-over-union includes the Q k actual intersection-over-union between the -1th sample face bounding box and the sample face bounding box with the maximum confidence in the Q k actual intersection-over-union between the -1th sample face bounding box and the sample face bounding box with the maximum confidence in the Q in a case where the loss function in the k th round satisfies a preset convergence condition, the training of the initial recognition model to be trained is ended, and the initial recognition model obtained in the k-1 th round of training is determined as the target recognition model; in a case where the loss function in the k th round does not satisfy the convergence condition, parameters in the initial recognition model obtained in the k-1 th round of training are adjusted to obtain the initial recognition model obtained in the k th round of training.

5. The method of claim 4, wherein, the loss function in the k th round is determined according to the k th set of actual intersection over unions used in the k th round and the k th set of predicted intersection over unions, including: According to the first set of actual intersection-over-union ratios and the second set of actual intersection-over-union ratios, a set of cross-entropy loss values is obtained, wherein the first set of actual intersection-over-union ratios includes W actual intersection-over-union ratios, the second set of actual intersection-over-union ratios includes W actual intersection-over-union ratios, the set of cross-entropy loss values includes W cross-entropy loss values, an i-th actual intersection-over-union ratio in the first set of actual intersection-over-union ratios is an actual intersection-over-union ratio between an i-th sample face bounding box in the Q k -1 sample face bounding boxes and a sample face bounding box with the highest confidence in the Q k -1 sample face bounding boxes, an i-th actual intersection-over-union ratio in the second set of actual intersection-over-union ratios is an actual intersection-over-union ratio between an i-th sample face bounding box in the Q k -1 sample face bounding boxes, an i-th cross-entropy loss value in the W cross-entropy loss values is obtained by performing cross-entropy loss operation on the i-th actual intersection-over-union ratio in the first set of actual intersection-over-union ratios and the i-th actual intersection-over-union ratio in the second set of actual intersection-over-union ratios, and W is equal to Q k -1, i is a positive integer greater than or equal to 1 and less than or equal to Q k -1. k ​ the loss function in the k th round is determined as an average value of the W actual intersection over unions.

6. An image detection apparatus characterized by comprising: including: a first input module configured to acquire N candidate face bounding boxes detected in a target image and N initial confidences, N being a positive integer greater than or equal to 1, the N candidate face bounding boxes and the N initial confidences having a one-to-one correspondence relationship; an iteration processing module, configured to perform iteration processing on the N candidate face detection boxes until a reserved candidate face detection box set is empty, wherein the iteration processing includes M rounds of processing, M candidate face detection boxes are extracted through the M rounds of processing, M is a positive integer greater than or equal to 1, a candidate face detection box extracted in an i-th round is extracted from a candidate face detection box set reserved in an (i-1)-th round according to a confidence set updated in the (i-1)-th round, in the i-th round of processing, if a candidate face detection box needs to be deleted from the candidate face detection box set reserved in the (i-1)-th round, the deleted candidate face detection box is a candidate face detection box determined according to a confidence set updated in the i-th round, the confidence set updated in the (i-1)-th round includes a confidence of each candidate face detection box in the candidate face detection box set reserved in the (i-1)-th round updated in the (i-1)-th round, when i=1, the confidence set updated in the (i-1)-th round includes the N initial confidences, the candidate face detection box set reserved in the (i-1)-th round includes the N candidate face detection boxes, i is a positive integer greater than or equal to 1 and less than or equal to M; a processing module, configured to determine the M candidate face detection boxes as results of face detection in the target image, and display the M candidate face detection boxes in the target image; wherein the iteration processing module is configured to perform the i-th round of processing through the following units: an extraction unit, configured to extract a corresponding candidate face detection box from the candidate face detection box set reserved in the (i-1)-th round to obtain a candidate face detection box set to be determined in the i-th round, wherein in the confidence set updated in the (i-1)-th round, a confidence of the candidate face detection box extracted in the i-th round is the highest; a determination unit, configured to determine an intersection over union between each candidate face detection box in the candidate face detection box set reserved in the (i-1)-th round except the candidate face detection box extracted in the i-th round and the candidate face detection box extracted in the i-th round to obtain an intersection over union set in the i-th round; an update unit, configured to update confidences in the confidence set updated in the (i-1)-th round except the confidence of the candidate face detection box extracted in the i-th round by using the intersection over union set in the i-th round to obtain a confidence set updated in the i-th round; and a deletion unit, configured to determine a confidence less than a preset threshold in the confidence set updated in the i-th round, and delete a candidate face detection box corresponding to the confidence less than the preset threshold in the candidate face detection box set to be determined in the i-th round to obtain a candidate face detection box set reserved in the i-th round. The updating unit is further configured to: in a case where the candidate face bounding box set reserved in the i-1th round of iterations includes P candidate face bounding boxes other than the candidate face bounding box extracted in the i th round of iterations, the confidence set updated in the i-1th round of iterations includes P confidences other than the confidence of the candidate face bounding box extracted in the i th round of iterations, the P confidences include the confidence of each of the P candidate face bounding boxes in the i-1th round of iterations, and the i th round of intersection-over-union sets includes P intersection-over-union ratios, in a case where the P intersection-over-union ratios are intersection-over-union ratios between the candidate face bounding box extracted in the i th round of iterations and the P candidate face bounding boxes respectively, the P intersection-over-union ratios are used to update a j th confidence in the P confidences by performing the following steps, where the j th intersection-over-union ratio is an intersection-over-union ratio between the candidate face bounding box extracted in the i th round of iterations and a j th candidate face bounding box in the P candidate face bounding boxes, and the j th confidence is the confidence of the j th candidate face bounding box updated in the i-1th round of iterations, P is a positive integer greater than or equal to 1, and j is a positive integer greater than or equal to 1 and less than or equal to P: determining an update factor of the j th confidence according to the j th intersection-over-union ratio; and determining a confidence of the j th candidate face bounding box updated in the i th round of iterations according to the update factor of the j th confidence, the j th confidence, and the j th intersection-over-union ratio. The update factor of the j th confidence and the j th intersection-over-union ratio are negatively correlated, where the greater the j th intersection-over-union ratio is, the smaller the update factor of the j th confidence is. The update factor of the jth confidence is determined according to the jth intersection over union, including: determining the update factor of the jth confidence by the following formula: wherein, is the update factor of the jth confidence, is the jth intersection over union.

7. A processor, comprising: The processor is configured to execute a program, and the program is configured to implement the method in any one of claims 1 to 5 when the program is executed.

8. An electronic device, comprising: The apparatus includes one or more processors and memory storing one or more programs, and the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method in any one of claims 1 to 5.

Images