Face detection method and apparatus

Abstract

A face detection method and apparatus are disclosed. The method includes: determining a detection box in an input image; calculating a confidence score indicating whether an object in the detection box corresponds to a face; setting an adaptive threshold based on the size of the detection box; and determining whether an object in the detection box corresponds to a face by comparing the confidence score with the adaptive threshold.

Description

[0001] This application claims the benefit of U.S. Provisional Application No. 62 / 934,594, filed November 13, 2019 with the United States Patent and Trademark Office, and the benefit of Korean Patent Application No. 10-2020-0027948, filed March 5, 2020 with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to methods and devices for face detection using adaptive thresholds. Background Technology

[0003] Face detection can be achieved using various techniques. For example, if the image is input via a camera, preprocessing such as resizing can be performed on the input image, and the preprocessed input image can be fed into a convolutional neural network (CNN). Candidate face boxes and their confidence scores can be determined based on the CNN's output feature maps, and candidate face boxes with confidence scores exceeding a threshold are identified as true faces. Face detection can be used in various fields, such as camera control and face recognition. Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, the selection of concepts that will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a face detection method is provided, the face detection method comprising: determining a detection box in an input image; calculating a confidence score indicating whether an object in the detection box corresponds to a face; setting an adaptive threshold based on the size of the detection box; and determining whether an object in the detection box corresponds to a face by comparing the confidence score with the adaptive threshold.

[0006] The setup steps may include setting an adaptive threshold based on inputting the size of the detection box into the threshold setting engine.

[0007] The threshold setting engine can determine the distribution of sample values ​​that indicate sample confidence scores based on the size of the sample detection box.

[0008] Each sample value can correspond to a true positive sample or a false positive sample, and the threshold setting engine can be configured to set an adaptive threshold using a function that distinguishes between true positive samples and false positive samples in the distribution.

[0009] Within the size range of the detection box, the adaptive threshold decreases in response to a decrease in the size of the detection box.

[0010] The setup steps may include setting an adaptive threshold based on the size of the detection box and the quality of the input image.

[0011] Face detection methods may include calculating the quality of the input image.

[0012] The setup steps may include setting an adaptive threshold by inputting the values ​​of the detection box size and the quality of the input image into the threshold setting engine.

[0013] The quality value can be used as a weight to input into the threshold setting engine.

[0014] The weights can be configured such that the adaptive threshold for low-quality images is set lower than the adaptive threshold for high-quality images.

[0015] The steps for calculating the quality of an input image may include: generating a downsampled image of the input image; generating a reference image corresponding to the size of the input image by interpolating the downsampled image; and calculating the quality of the input image based on the difference between the input image and the reference image.

[0016] The quality of an input image can be determined based on any one or any combination of the input image's resolution, blur level, and noise level.

[0017] Face detection methods may include determining whether an object in the detection box corresponds to a face by comparing a confidence score with at least one of a maximum threshold or a minimum threshold.

[0018] The face detection method may also include: in response to determining that a face bounding box may exist in the input image, performing any one or any combination of camera control and face recognition.

[0019] In another general aspect, a face detection device is provided, the face detection device comprising: a processor configured to: determine a detection box in an input image; calculate a confidence score indicating whether an object in the detection box corresponds to a face; set an adaptive threshold based on the size of the detection box; and determine whether an object in the detection box corresponds to a face by comparing the confidence score with the adaptive threshold.

[0020] The processor can be configured to set an adaptive threshold based on the value of the detection box size being input into the threshold setting engine.

[0021] The threshold setting engine can determine the threshold based on the distribution of sample values ​​that indicate the confidence score of the sample, according to the size of the sample detection box.

[0022] Within the size range of the detection box, the adaptive threshold decreases in response to a decrease in the size of the detection box.

[0023] The processor can be configured to additionally set an adaptive threshold based on the quality of the input image.

[0024] The processor can be configured to set an adaptive threshold by inputting the size value of the detection box and the quality value of the input image into the threshold setting engine, and the quality value can be input into the threshold setting engine as a weight.

[0025] The processor can be configured to generate an early detection result indicating whether an object in the detection box corresponds to a face by comparing the confidence score with at least one of a maximum threshold or a minimum threshold.

[0026] The processor can be configured to use a compensation value to increase the confidence score of low-quality images or decrease the confidence score of high-quality images in response to an increase in the size of the detection box.

[0027] The processor can be configured to use a compensation value to increase the confidence score of low-quality images or decrease the confidence score of high-quality images in response to an increase in the size of the detection box.

[0028] The compensation value can be based on any one or any combination of the size of the detection box and the quality of the input image.

[0029] A face detection device may include a memory configured to store instructions that can be executed by a processor.

[0030] In another general aspect, a face detection device is provided, the face detection device comprising: a sensor configured to receive an image; and a processor configured to: determine a detection box in the image; calculate a confidence score indicating whether an object in the detection box corresponds to a face; determine that an object in the detection box corresponds to a face in response to a confidence score greater than a maximum threshold; set an adaptive threshold based on the size of the detection box; and determine whether an object in the detection box corresponds to a face based on a comparison of the confidence score and the adaptive threshold in response to a confidence score less than the maximum threshold.

[0031] The face detection device may also include: discarding the detection frame in response to a confidence score less than a minimum threshold.

[0032] Other features and aspects will become clear from the following detailed description and accompanying drawings. Attached Figure Description

[0033] Figure 1 An example of the operation of a face detection device is shown.

[0034] Figure 2 An example is shown showing the probability of false detection based on a fixed threshold.

[0035] Figure 3Examples of detection boxes, including faces and non-faces, are shown by size.

[0036] Figure 4 Examples of detection boxes of various sizes are shown in high-quality and low-quality input images.

[0037] Figure 5 An example of a face detection method is shown.

[0038] Figure 6 This shows an example of how the threshold setting engine operates.

[0039] Figure 7 This shows an example of how the design threshold setting engine processes data.

[0040] Figure 8 This shows an example of how the design threshold setting engine processes data.

[0041] Figure 9 An example of quality calculation processing is shown.

[0042] Figure 10 An example of a face detection method is shown.

[0043] Figure 11 An example of a face detection device is shown.

[0044] Figure 12 An example of an electronic device is shown.

[0045] Figure 13 An example of an electronic device is shown.

[0046] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated. Detailed Implementation

[0047] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various variations, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order of operations set forth herein, but may be changed as will become apparent after understanding the disclosure of this application, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0048] The features described herein can be implemented in various forms and should not be construed as limited to the examples described herein. Rather, the examples provided herein illustrate only a few of the many possible ways in which the methods, apparatus, and / or systems described herein will become clear upon understanding the disclosure of this application.

[0049] Although the terms “first,” “second,” A, B, (a), (b), etc., are used to explain various components, the components are not limited by the terms. These terms should only be used to distinguish one component from another. For example, without departing from the teaching of the examples, the first component may be referred to as the second component, and similarly, the second component may be referred to as the first component.

[0050] Throughout this specification, when a component is described as being "connected to" or "bonded to" another component, that component may be directly "connected to" or directly "bonded to" another component, or there may be one or more other components in between. In contrast, when an element is described as being "directly connected to" or "directly bonded to" another element, there may not be any other elements in between. Similarly, similar expressions, such as "between" and "immediately between," "adjacent to" and "right next to," should be interpreted in the same manner. As used herein, the term "and / or" includes any one and any combination of any two or more of the associated listed items.

[0051] As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. It will also be understood that the terms “comprising” and / or “including” as used herein indicate the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof.

[0052] The examples will be described in detail below with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same elements.

[0053] Figure 1 An example of the operation of a face detection device is shown. (See reference) Figure 1 The face detection device 100 receives an input image 110. The face detection device 100 can detect detection boxes 111 that include objects 112 estimated to be faces in the input image 110, and determine whether the objects 112 in the detection boxes 111 correspond to faces. If the objects 112 in the detection boxes 111 correspond to faces, the face detection device 100 can identify the detection boxes 111 as face boxes. If the objects 112 in the detection boxes 111 do not correspond to faces, the detection boxes 111 can be discarded.

[0054] If a face bounding box is determined, the face detection device 100 outputs a detection result 120 corresponding to the face bounding box. The detection result 120 may include face bounding box information, and the face bounding box can be specified in the input image 110 using this information. For example, the face bounding box information may include reference coordinate values ​​of the face bounding box (e.g., the coordinates of a vertex of the face bounding box) and size values ​​of the face bounding box (e.g., the diagonal length of the face bounding box). The face bounding box information is not limited to this, and the face bounding box can be specified in various ways. The detection result 120 can be used to control the camera that generates the input image 110. For example, camera control may include autofocus, auto exposure, and auto white balance, referred to as 3A.

[0055] The face detection device 100 generates a detection result 120 by comparing a confidence score 113 with an adaptive threshold 116. When using a fixed threshold, it can be difficult to suppress false detections, as described below. False detections may include incorrectly detecting a face as a non-face (hereinafter referred to as a false negative) and incorrectly detecting a non-face as a face (hereinafter referred to as a false positive). The adaptive threshold 116 can be adjusted based on the bounding box size and / or image quality, and false detections can be significantly suppressed by the adaptive threshold 116.

[0056] In one example, face detection device 100 can calculate a confidence score 113 for a detection box 111, compare the confidence score 113 with an adaptive threshold 116, and generate a detection result 120 based on the comparison result. The confidence score 113 can indicate whether an object 112 in the detection box 111 corresponds to a face. For example, the confidence score 113 can indicate the probability that object 112 corresponds to a face. A neural network-based face detector can be used for the detection of the detection box 111 and the calculation of the confidence score 113. In one example, a convolutional neural network (CNN) can be used.

[0057] The face detection device 100 can set an adaptive threshold 116 based on the size 114 of the detection frame 111. The confidence score 113 can decrease as the size 114 decreases, as will be described below. Based on this characteristic, the face detection device 100 can set an adaptive threshold 116 suitable for the size 114. For example, when the size 114 is small within the size range, the face detection device 100 can set the adaptive threshold 116 to be small. For example, the smaller the size 114 is within the size range, the smaller the adaptive threshold 116.

[0058] In another example, the face detection device 100 may set an adaptive threshold 116 based on the size 114 and quality 115 of the detection box 111. For example, the quality 115 may be determined based on any one or any combination of resolution, blur level, and noise level. The quality 115 may be associated with the input image 110 or with the detection box 111. Hereinafter, the quality 115 will be representatively described as being associated with the input image 110. Furthermore, the confidence score 113 may decrease as the quality 115 decreases, as will be described below. Based on this characteristic, the face detection device 100 may set an adaptive threshold 116 suitable for the quality 115.

[0059] The quality 115 can be reflected in the adaptive threshold 116 in various ways. In one example, the adaptive threshold 116 can be set via a pre-designed threshold setting engine. The threshold setting engine can be designed based on the distribution of sample scores according to the sample's bounding box size. Samples can include facial samples and non-facial samples.

[0060] For example, sample quality can be reflected as a compensation value in the score and affect the distribution of scores based on the box size used to design the thresholding engine. In this example, when a detection result of 120 is obtained using a pre-designed thresholding engine, the quality of 115 is not separately input into the thresholding engine.

[0061] In another example, the sample quality of each sample may not be reflected in the distribution, but the weight corresponding to the sample quality can be applied as a variable to the thresholding engine. In this example, when the pre-designed thresholding engine produces a detection result of 120, the quality 115 can be input into the thresholding engine as a weight. For example, as the quality 115 decreases, a larger weight can be applied to the thresholding engine.

[0062] Figure 2 An example is shown illustrating the probability of false detection based on a fixed threshold. (See reference...) Figure 2Plot 210 shows the distribution of confidence scores for non-face samples, and plot 220 shows the distribution of confidence scores for face samples. In this example, threshold 231 can be set based on the distribution of confidence scores represented by plots 210 and 220. Threshold 231 can be set to a fixed value. For example, threshold 231 can be set to a fixed value corresponding to the confidence scores at the overlap of plots 210 and 220. In this example, face samples belonging to region 211 may have confidence scores below threshold 231, thus being incorrectly detected as non-faces. Non-face samples belonging to region 221 may have confidence scores above threshold 231, thus being incorrectly detected as faces. The likelihood of such false detections can be greatly reduced if threshold 231 is specifically adjusted to suit the situation based on bounding box size or quality.

[0063] Figure 3 Examples of detection boxes, including facial and non-facial areas, are shown in size. (See reference.) Figure 3 Detection boxes 311, 312, and 313 correspond to faces, while detection boxes 321, 322, and 323 correspond to non-faces. For example, the confidence score corresponding to a specific box size can be defined as a size-score pair, which can be expressed as (box size, confidence score). For example, through experiments, the size-score pairs for detection boxes 311, 312, and 313 can be obtained as (238, 0.987), (175, 0.98), and (112, 0.951), and the size-score pairs for detection boxes 321, 322, and 323 can be obtained as (250, 0.772), (205, 0.743), and (167, 0.669).

[0064] In all the detection boxes 311, 312, and 313 corresponding to faces and 321, 322, and 323 corresponding to non-faces, the confidence scores are shown to increase with increasing box size. Therefore, the adaptive threshold can typically be set to increase with increasing box size. Furthermore, when comparing face boxes and non-face boxes of similar size, face boxes have higher confidence scores than non-face boxes. For example, the confidence score of detection box 311 is greater than that of detection box 321. Similar results can be obtained from comparisons between detection boxes 312 and 322, and between detection boxes 313 and 323. Therefore, setting the adaptive threshold to increase with increasing box size effectively suppresses false detections.

[0065] Figure 4 Examples of detection boxes of various sizes are shown in both high-quality and low-quality input images. (See reference...) Figure 4Detection boxes 411, 412, and 413 were detected in the high-quality input image 410, while detection boxes 421, 422, and 423 were detected in the low-quality input image 420. Image quality can be calculated in various ways, and low-resolution images, blurry images, and noisy images can be classified as low-quality images. For example, through experiments, the confidence scores for detection boxes 411, 412, and 413 were found to be 0.897, 0.871, and 0.835, respectively, and the confidence scores for detection boxes 421, 422, and 423 were found to be 0.764, 0.827, and 0.799, respectively.

[0066] When comparing the confidence scores of detection boxes 411, 412, and 413 corresponding to high-quality images with the confidence scores of detection boxes 421, 422, and 423 corresponding to low-quality images by size, the boxes corresponding to high-quality images have higher confidence scores among boxes of similar size. Therefore, appropriate compensation can be provided to increase the confidence score of low-quality images or decrease the confidence score of high-quality images. If weights are used, they can be designed such that the adaptive threshold for low-quality images can be set relatively low, or the adaptive threshold for high-quality images can be set relatively high.

[0067] Furthermore, as the bounding box size increases, the difference between the confidence scores of high-quality images and low-quality images can increase significantly. Therefore, compensation values ​​can be used to further increase the confidence score of low-quality images or further decrease the confidence score of high-quality images as the bounding box size increases. If weights are used, they can be designed so that the adaptive threshold for low-quality images can be further set lower, or the adaptive threshold for high-quality images can be further set higher.

[0068] Figure 5 An example of a face detection method is shown. Figure 5 The operations can be performed in the order and manner shown, although the order of some operations may be changed or some operations may be omitted without departing from the spirit and scope of the exemplary examples described. Figure 5 Many of the operations shown can be performed in parallel or simultaneously. Figure 5 One or more boxes, and combinations thereof, can be implemented by a computer (such as a processor) based on dedicated hardware that performs a specific function, or by a combination of dedicated hardware and computer instructions. In addition to the following... Figure 5 In addition to the description, Figures 1 to 4 The description also applies to Figure 5 And it is included here by reference. Therefore, the above description need not be repeated here.

[0069] Reference Figure 5In operation 510, the face detection device receives an input image. For example, the input image may include multiple frames; here, the first frame of the input image is received. As described below, after processing the first frame, the second and third frames can be processed sequentially.

[0070] In operation 520, the face detection device determines detection boxes in the input image. The detection boxes may include objects estimated to be faces. In operation 530, the face detection device calculates a confidence score for each detection box. The confidence score indicates whether an object within the detection box corresponds to a face. For example, the confidence score indicates the probability that an object corresponds to a face.

[0071] In one example, a face detection device can use a face detector based on a trained neural network to determine bounding boxes and calculate confidence scores. For example, the face detector may include a CNN. The face detection device may apply preprocessing (such as resizing) to the input image and feed the preprocessed input image into the face detector. Therefore, the face detector can output bounding boxes and their confidence scores.

[0072] In operation 540, the face detection device sets an adaptive threshold for the detection bounding box. For example, within a range of detection box sizes, the adaptive threshold can be set to decrease as the size of the detection box decreases. The size range can represent a range from the size corresponding to the minimum threshold to the size corresponding to the maximum threshold. The face detection device can set the adaptive threshold based on the size of the detection box, or it can set the adaptive threshold based on the size of the detection box and the quality of the input image. Therefore, different thresholds can be applied to each detection box based on the box size and / or image quality. Operations 530 and 540 can be performed simultaneously, or one of operations 530 and 540 can be performed first.

[0073] Face detection devices can use a thresholding engine to set adaptive thresholds. The thresholding engine can be determined based on the distribution of multiple sample values ​​indicating confidence scores, each corresponding to the size of the detection bounding box. The face detection device can input the size values ​​of the detection bounding boxes as input data to the thresholding engine, and / or input the quality values ​​of the input image as input data to the thresholding engine.

[0074] For example, the quality value of the input image can be used as a weight for setting the threshold. For instance, the weight can be designed such that the adaptive threshold for low-quality images can be set relatively low, or the adaptive threshold for high-quality images can be set relatively high. The weight can also be designed such that the adaptive threshold for low-quality images can be set relatively low, and the adaptive threshold for high-quality images can be set relatively high. The threshold setting engine will be described in detail later.

[0075] In operation 550, the face detection device compares a confidence score with an adaptive threshold to determine whether an object in the detection box corresponds to a face. If the confidence score is less than the adaptive threshold, operation 560 is performed. In operation 560, the face detection device discards the detection box. If the confidence score is greater than the adaptive threshold, operation 570 is performed. In operation 570, the face detection device identifies the detection box as a face box.

[0076] In response to determining the presence of a face bounding box in the input image via operation 570, various post-processing techniques associated with the face bounding box can be performed. For example, post-processing may include camera control and face recognition. For instance, autofocus, auto exposure, and auto white balance, referred to as 3A, may be performed based on the face bounding box. After operations 560 and 570 are performed, operation 510 may be performed on a subsequent frame (e.g., a second frame).

[0077] Figure 6 An example of the operation of a threshold setting engine is shown. The threshold setting engine 610 may be designed to set an adaptive threshold 613 suitable for the input data. The input data may include the size 611 of the bounding box, or it may include the size 611 of the bounding box and the quality 612 of the image. For example, the threshold setting engine 610 may determine an adaptive threshold 613 suitable for size 611 in response to the input of size 611. In another example, the threshold setting engine 610 may determine an adaptive threshold 613 suitable for size 611 and quality 612 in response to the input of size 611 and quality 612.

[0078] The threshold setting engine 610 can be pre-designed based on the distribution of multiple sample values, each indicating a sample confidence score according to the size of the sample detection box. The threshold setting engine 610 can be designed to include a function (e.g., a linear or exponential function) for distinguishing between facial and non-facial samples in the distribution of sample values. In another example, the threshold setting engine 610 can be designed by calculating thresholds for multiple intervals and performing interpolation between the thresholds. The process of designing the threshold setting engine 610 will be described in further detail below.

[0079] Figure 7 This illustrates an example of how the design threshold setting engine processes data. (See reference...) Figure 7 This shows the points corresponding to multiple sample values. Sample values ​​can correspond to true positive or false positive samples. If a detection box for a sample that includes a face is identified as a face box, then that sample corresponds to a true positive. When a detection box for a sample that does not include a face (i.e., includes non-face areas) is identified as a face box, that sample corresponds to a false positive. Each sample value can include a size value and a confidence score. Therefore, Figure 7 The points in the diagram indicate the distribution of sample confidence scores based on the size of the sample detection box.

[0080] A function can be defined to distinguish between true positive samples and false positive samples in this distribution. For example, the function may include a first-order function and an exponential function. Curves 710 and 720 distinguish between true positive samples and false positive samples in a given interval AB, respectively. Curve 710 may correspond to a first-order function, and curve 720 may correspond to an exponential function. The first-order function can be defined by Equation 1, and the exponential function can be defined by Equation 2.

[0081] [Equation 1]

[0082] y = ax + b

[0083] [Equation 2]

[0084]

[0085] In Equations 1 and 2, x represents the size and y represents the confidence score. The values ​​of a and b can be determined such that true positives and false positives can be distinguished based on a given performance index (e.g., false acceptance rate (FAR)). Since true positives exhibit a curvilinear distribution, an exponential function may be advantageous if a high performance index is required.

[0086] Functions can be defined for ranges (e.g., interval AB). For example, if the coordinates of point A are (20, 0.65) and the coordinates of point B are (200, 0.95), then a function can be defined for a size range of 20 to 200. This size range can correspond to a confidence score range of 0.65 to 0.95. In this example, the minimum and maximum confidence scores within the range in which the function is defined can indicate the maximum and minimum thresholds, respectively. For example, in... Figure 7 In this context, 0.65 indicates the minimum threshold, and 0.95 indicates the maximum threshold.

[0087] Maximum and minimum thresholds can be used to generate early detection results, which will be described later. Detection boxes with confidence scores less than the minimum threshold indicate that the detection box is very unlikely to correspond to a face, and therefore can be discarded immediately without considering the adaptive threshold. Conversely, detection boxes with confidence scores greater than the maximum threshold indicate that the detection box is very likely to correspond to a face, and therefore can be immediately identified as face boxes without considering the adaptive threshold.

[0088] When designing a threshold setting engine, sample quality can be considered. For example, sample quality can be used to compensate for confidence scores. In this example, a quality-based compensation value can be applied to the confidence score of each sample. By applying the compensation value, the distribution of samples can be adjusted. For example, Figure 7 This shows the distribution before compensation is reflected, and how it can be adjusted if compensation is reflected. Figure 7The distribution in the adjusted distribution makes it easier to distinguish between true positive and false positive samples. In other words, applying the compensation value makes it even easier to differentiate between true positive and false positive samples.

[0089] When setting compensation values, the quality level can be considered. For example, the compensation value can be set to increase the confidence score of low-quality images or decrease the confidence score of high-quality images. Since low quality can lead to low confidence scores, this result can be mitigated by adjusting the confidence score via the compensation value. The compensation value can be set to increase the confidence score of low-quality images and decrease the confidence score of high-quality images.

[0090] Furthermore, the frame size can be used additionally when setting compensation values. This is because the difference between the confidence scores of high-quality and low-quality images can increase significantly as the frame size increases. For example, the compensation value can be set to further increase the confidence score of low-quality images or further decrease the confidence score of high-quality images as the frame size increases. The compensation value can also be designed to further increase the confidence score of low-quality images and further decrease the confidence score of high-quality images as the frame size increases.

[0091] A compensation function can be defined to set the compensation value. The compensation function can receive a mass value or a mass value and a frame size value, and output the compensation value. The above principles can be considered when designing the compensation function.

[0092] In another example, the quality of the samples can be used as a threshold to set the weights of the engine. In this example, equations 1 and 2 can be modified into equations 3 and 4.

[0093] [Equation 3]

[0094] y = w(ax + b)

[0095] [Equation 4]

[0096]

[0097] In Equations 3 and 4, w represents the weight. The weight can be set such that the adaptive threshold for low-quality images can be set relatively low, or the adaptive threshold for high-quality images can be set relatively high. Since low quality can lead to low confidence scores, this can be mitigated by adjusting the threshold via the weight. The weight can also be set such that the adaptive threshold for low-quality images can be set relatively low, and the adaptive threshold for high-quality images can be set relatively high.

[0098] The weights can be set such that the adaptive threshold for low-quality images can be set lower, or the adaptive threshold for high-quality images can be set higher. This is because the difference between the confidence scores of high-quality and low-quality images can increase as the bounding box size increases. The weights can be set such that the adaptive threshold for low-quality images can be set lower, and the adaptive threshold for high-quality images can be set higher.

[0099] A weighting function can be defined to set the weights. The weighting function can receive a quality value or a quality value and a box size value, and output the weight value. The above principles can be considered when designing the weighting function. It should be understood that the function defined in this application for distinguishing true positive samples and false positive samples in this distribution is not limited to first-order functions and exponential functions; other functions capable of distinguishing true positive samples and false positive samples in this distribution are also feasible.

[0100] Figure 8 This illustrates an example of how the design threshold setting engine processes data. (See reference...) Figure 8 A reference threshold for the reference size can be determined, and a function can be defined based on the interpolation associated with the reference threshold. Graph 800 corresponds to the function defined through this process.

[0101] For example, for reference sizes 10, 20, ..., 10n, a first threshold, a second threshold, ..., an nth threshold can be determined based on the reference size. Each threshold can be determined based on a given performance index (e.g., FAR). For example, graphs 810, 820, and 830 are shown for the reference sizes, and the corresponding threshold can be determined based on the size of the region where error detection occurs. Interpolation can be performed based on the thresholds to define a function corresponding to graph 800. The above description of quality can also be applied to... Figure 8 Examples include: for instance, a function can be defined after quality data has been pre-reflected (e.g., compensated) in the confidence score, or the quality data can be applied as weights to the function after it has been defined.

[0102] Figure 9 An example of quality calculation processing is shown. (See reference...) Figure 9 A downsampled image 920 is generated by downsampling the input image 910, and a reference image 930 is generated by interpolating the downsampled image 920. In one example, nearest-neighbor interpolation is performed. The size of the reference image 930 may correspond to the size of the input image 910. The quality of the input image 910 can be calculated based on the difference between the input image 910 and the reference image 930. For example, the difference can be calculated using the L1 norm.

[0103] The greater the difference, the higher the quality of input image 910 can be determined. For example, input image 911 may correspond to a high-quality image, and input image 912 may correspond to a low-quality image. In this example, the difference between input image 911 and reference image 931 may be greater than the difference between input image 912 and reference image 932. This is because a greater information loss occurs due to downsampling. Therefore, input image 911 with a relatively large difference can be determined as a relatively high-quality image, and input image 912 with a relatively small difference can be determined as a relatively low-quality image. Quality calculation is not necessarily limited to the above example and can be performed in various other ways.

[0104] Figure 10 An example of a face detection method is shown. Figure 10 The operations can be performed in the order and manner shown, although the order of some operations may be changed or some operations may be omitted without departing from the spirit and scope of the exemplary examples described. Figure 10 Many of the operations shown can be performed in parallel or simultaneously. Figure 10 One or more boxes, and combinations thereof, can be implemented by a computer (such as a processor) based on dedicated hardware that performs a specific function, or by a combination of dedicated hardware and computer instructions. In addition to the following... Figure 10 In addition to the description, Figures 1 to 9 The description also applies to Figure 10 And it is included here by reference. Therefore, the above description need not be repeated here.

[0105] As described above, early detection results can be generated by comparing the confidence score with at least one of a preset maximum threshold and a preset minimum threshold. Early detection results can be compared with... Figure 1 The detection result 120 is different and can be generated earlier than the detection result 120. For example, a detection box with a confidence score less than the minimum threshold indicates that the detection box is very unlikely to correspond to a face, and therefore can be discarded immediately without considering the adaptive threshold. Conversely, a detection box with a confidence score greater than the maximum threshold indicates that the detection box is very likely to correspond to a face, and therefore can be immediately identified as a face box without considering the adaptive threshold. In detail, the early detection results can be generated through the operations described below.

[0106] Reference Figure 10In operation 1010, the face detection device receives an input image, and in operation 1020, it determines detection boxes in the input image. In operation 1030, the face detection device calculates a confidence score for the detection boxes. In operations 1050 and 1060, the face detection device attempts to generate early detection results. If the confidence score is less than a minimum threshold, the face detection device may discard the detection box in operation 1080. If the confidence score is greater than a maximum threshold, the face detection device may identify the detection box as a face box in operation 1090.

[0107] In operation 1040, the face detection device sets an adaptive threshold for the detection frame, and in operation 1070, determines whether an object included in the detection frame corresponds to a face by comparing a confidence score with the adaptive threshold. If the confidence score is less than the adaptive threshold, operation 1080 can be executed. If the confidence score is greater than the adaptive threshold, operation 1090 can be executed. If either operation 1050 or 1060 is determined to be true, operations 1040 and 1070 can be omitted. If a detection result is generated through operation 1070, the detection result can correspond to... Figure 1 The detection result is 120. If the detection result is not generated through operation 1070, the detection result can correspond to an earlier detection result. Furthermore, refer to... Figure 5 The provided description is applicable Figure 10 Facial detection methods.

[0108] Figure 11 An example of a face detection device is shown. (See reference) Figure 11 The face detection device 1100 includes a processor 1110 and a memory 1120. The memory 1120 is connected to the processor 1110 and can store instructions executable by the processor 1110, data to be calculated by the processor 1110, or data processed by the processor 1110. The memory 1120 may include non-transitory computer-readable media (such as high-speed random access memory) and / or non-volatile computer-readable storage media (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0109] Processor 1110 is executable for performing reference Figures 1 to 10 Instructions describing one or more operations. For example, processor 1110 may determine a detection box in an input image, calculate a confidence score indicating whether an object in the detection box corresponds to a face, set an adaptive threshold based on the size of the detection box, and determine whether an object in the detection box corresponds to a face by comparing the confidence score with the adaptive threshold.

[0110] Figure 12 An example of an electronic device is shown. (See reference) Figure 12The electronic device 1200 may include a processor 1210, a memory 1220, a camera 1230, a storage device 1250, an input device 1260, an output device 1270, and a network interface 1280. The processor 1210, memory 1220, camera 1230, storage device 1250, input device 1260, output device 1270, and network interface 1280 may communicate with each other via a communication bus 1290.

[0111] For example, electronic device 1200 may be implemented as at least a part of mobile devices (such as mobile phones, smartphones, PDAs, netbooks, tablet computers, or laptop computers), wearable devices (such as smartwatches, smart bracelets, or smart glasses), electronic products (such as robots, digital cameras, digital camcorders, portable game consoles, MP3 players, portable / personal multimedia players (PMPs), handheld e-readers, GPS navigation, personal navigation devices, portable navigation devices (PNDs), handheld game consoles, e-readers, various Internet of Things (IoT) devices), computing devices (such as desktop computers or servers), home appliances (such as televisions, smart TVs, refrigerators, smart home devices), security devices (such as door locks), or vehicles (such as intelligent vehicles, automatic or autonomous driving systems, advanced driver assistance systems (ADAS)), or any other device capable of wireless or network communication consistent with the contents disclosed herein.

[0112] Electronic device 1200 can acquire an input image and detect face bounding boxes from the acquired input image. Furthermore, electronic device 1200 can perform operations associated with the detected face bounding boxes. Electronic device 1200 may structurally and / or functionally include... Figure 1 A facial detection device 100. For example, electronic device 1200 may functionally include a processor 1210 and a memory 1220. Figure 1 The facial detection device 100, or may structurally include a... Figure 1 The face detection device 100 corresponds to the face detection device 1240.

[0113] Processor 1210 executes instructions or functions that will be performed in electronic device 1200. For example, processor 1210 may process instructions stored in memory 1220 or storage device 1240. Processor 1210 may execute instructions via... Figures 1 to 11 One or more operations are described. Further details about the processor 1210 are provided below.

[0114] Memory 1220 stores data used for face detection. Memory 1220 may include a computer-readable storage medium or a computer-readable storage device. Memory 1220 may store instructions to be executed by processor 1210, and may store related information when software and / or applications are executed by electronic device 1200. Further details regarding memory 1220 are provided below.

[0115] Camera 1230 can capture photos and / or videos. For example, camera 1230 can capture facial images including the user's face. Camera 1230 can provide 3D images including depth information about objects.

[0116] Storage device 1250 includes a computer-readable storage medium or a computer-readable storage device. Storage device 1250 can store various data to be used in face detection processing (such as face detectors or threshold setting engines). Compared to memory 1220, storage device 1250 can store a greater amount of information for a longer period of time. For example, storage device 1250 may include a magnetic hard disk, optical disk, flash memory, floppy disk, or other non-volatile memory known in the art.

[0117] Input device 1260 can receive input from a user via a keyboard and mouse in traditional input methods as well as in new input methods (such as touch input, voice input, and image input). For example, input device 1260 may include a keyboard, mouse, touch screen, microphone, or any other device that detects input from the user and transmits the detected input to electronic device 1200.

[0118] Output device 1270 provides output from electronic device 1200 to a user via visual, auditory, or tactile channels. Output device 1270 may include, for example, a display, touchscreen, speaker, vibration generator, or any other device that provides output to the user. In one example, output device 1270 is a physical structure including one or more hardware components that provide the ability to render a user interface, render a display, and / or receive user input. Output device 1270 is not limited to the examples described above, and any other display operatively connected to electronic device 1200 (e.g., a computer monitor and an eyeglass display (EGD)) may be used without departing from the spirit and scope of the exemplary examples described. Network interface 1280 can communicate with external devices via wired or wireless networks.

[0119] Figure 13 An example of an electronic device is shown. (See reference) Figure 13The electronic device 1300 may include a processor 1310, a memory 1320, a camera 1330, a storage device 1350, an input device 1360, an output device 1370, and a network interface 1380. The processor 1310, memory 1320, camera 1330, storage device 1350, input device 1360, output device 1370, and network interface 1380 can communicate with each other via a communication bus 1390. (Except for the following...) Figure 13 In addition to the description, Figures 11 to 12 The description also applies to Figure 13 And it is included here by reference. Therefore, the above description need not be repeated here.

[0120] Camera 1330 may include face detection device 1340. For example, camera 1330 may include face detection device 1340 as one or more modules. Face detection results from face detection device 1340 can be used to control camera 1330. For example, control of camera 1330 may include autofocus, auto exposure, and auto white balance, referred to as 3A. Face detection device 1340 may include, for example, a processor (e.g., Figure 11 The processor 1110 can be used to generate face detection results. Furthermore, the camera 1330 can be controlled based on the face detection results generated as described above. Therefore, face detection and control of the camera 1330 can be performed without intervention from the processor 1310 or with minimal intervention from the processor 1310.

[0121] Face detection devices 100, 1100, 1240, and 1340, the threshold setting engine 610, and other devices, units, modules, apparatuses, and components described herein are implemented via hardware components. Examples of hardware components that can be used to perform the operations described herein include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described herein. In other examples, one or more of the hardware components performing the operations described herein are implemented via computing hardware (e.g., via one or more processors or computers). Processors or computers can be implemented via one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described herein. The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For the sake of brevity, the singular terms "processor" or "computer" are used in the description of the examples described herein; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component or two or more hardware components.The hardware components may be any one or more with different processing configurations. Examples of different processing configurations include: a single processor, a standalone processor, a parallel processor, a single instruction single data (SISD) multiprocessor, a single instruction multiple data (SIMD) multiprocessor, multiple instruction single data (MISD) multiprocessor, multiple instruction multiple data (MIMD) multiprocessor, a controller and arithmetic logic unit (ALU), a DSP, a microcomputer, an FPGA, a programmable logic unit (PLU), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or any other device capable of responding to and executing instructions in a defined manner.

[0122] The methods for performing the operations described in this application are executed by computing hardware (e.g., by one or more processors or a computer), which is implemented as described above to execute instructions or software to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations may be executed by a single processor or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation or two or more operations.

[0123] Instructions or software for controlling a processor or computer to implement hardware components and perform the methods described above are written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure the processor or computer to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include at least one of a applet storing a face detection method, a dynamic link library (DLL), middleware, firmware, a device driver, or an application program. In another example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by the processor or computer. In yet another example, the instructions or software include high-level code that is executed by the processor or computer using an interpreter. Those skilled in the art can readily write instructions or software based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0124] Instructions or software used to control a processor or computer to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, are recorded, stored, or fixed in, or on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), flash memory, card storage (such as multimedia microcards or cards (e.g., Secure Digital (SD) or Extreme Digital (XD)), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other means configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and provide said instructions or software and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored and executed in a distributed manner by one or more processors or computers.

[0125] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example will be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of this disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be construed as included in this disclosure.

Claims

1. A face detection method, comprising: Determine the detection boxes in the input image; Calculate the confidence score for whether the object in the detection box corresponds to a face; An adaptive threshold is set based on the size of the detection frame; as well as Whether an object in the detection box corresponds to a face is determined by comparing a confidence score with an adaptive threshold. The setup steps include: setting an adaptive threshold based on inputting the size of the detection box into the threshold setting engine. The threshold setting engine is determined based on the distribution of sample values ​​indicating sample confidence scores according to the size of the sample detection box. The quality of the sample is used to compensate for the sample confidence score in order to adjust the distribution of sample values. Each sample value corresponds to either a true positive sample or a false positive sample, and The threshold setting engine is configured to set an adaptive threshold using a function that distinguishes between true positive samples and false positive samples in the distribution.

2. The face detection method according to claim 1, wherein, The face detection method further includes: calculating the quality of the input image and applying a compensation value based on the quality of the input image to the confidence score. The compensation value is set to increase the confidence score of low-quality images and / or decrease the confidence score of high-quality images.

3. The face detection method according to claim 2, wherein, The compensation value is also set to further increase the confidence score of low-quality images as the size of the detection box increases, and / or further decrease the confidence score of high-quality images.

4. The face detection method according to any one of claims 1 to 3, wherein, Within the size range of the detection box, the adaptive threshold decreases in response to a decrease in the size of the detection box.

5. The face detection method according to any one of claims 1 to 3, wherein, The setup steps also include setting an adaptive threshold based on the quality of the input image.

6. The face detection method according to claim 1, wherein, The setup steps include setting an adaptive threshold by inputting the size of the detection box and the quality of the input image into the threshold setting engine.

7. The face detection method according to claim 6, wherein, The quality value is input as a weight into the threshold setting engine.

8. The face detection method according to claim 7, wherein, The weights are configured such that the adaptive threshold for low-quality images is set lower than the adaptive threshold for high-quality images.

9. The face detection method according to claim 1, wherein, The steps for calculating the quality of an input image include: Generate a downsampled image of the input image; A reference image corresponding to the size of the input image is generated by interpolating the downsampled image; and The quality of the input image is calculated based on the difference between the input image and the reference image.

10. The face detection method according to claim 1, wherein, The quality of an input image is determined based on any one or any combination of the input image's resolution, blur level, and noise level.

11. The face detection method according to any one of claims 1 to 3, further comprising: Whether an object in the detection box corresponds to a face is determined by comparing the confidence score with at least one of the maximum and minimum thresholds.

12. The face detection method according to claim 11, wherein, In response to a confidence score greater than the maximum threshold, it is determined that the object in the detection box corresponds to a face. In response to a confidence score less than a minimum threshold, it is determined that the object in the detection box does not correspond to a face, and the detection box is discarded. In response to a confidence score greater than the minimum threshold and less than the maximum threshold, the system determines whether an object in the detection box corresponds to a face by comparing the confidence score with an adaptive threshold.

13. The face detection method according to any one of claims 1 to 3, further comprising: In response to determining that an object in the detection box corresponds to a face, perform either or any combination of camera control and face recognition.

14. A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, cause the processor to perform the face detection method according to any one of claims 1 to 13.

15. A facial detection device, comprising: The processor is configured as follows: Determine the detection boxes in the input image. Calculate the confidence score indicating whether the object in the detection box corresponds to a face. An adaptive threshold is set based on the size of the detection box, and Whether an object in the detection box corresponds to a face is determined by comparing a confidence score with an adaptive threshold. The processor is configured to set an adaptive threshold based on the value of the detection box size input into the threshold setting engine. The threshold setting engine determines the threshold based on the distribution of sample values ​​indicating sample confidence scores according to the size of the sample detection box. The quality of the sample is used to compensate for the sample confidence score in order to adjust the distribution of sample values. Each sample value corresponds to either a true positive sample or a false positive sample. The threshold setting engine is configured to set an adaptive threshold using a function that distinguishes between true positive samples and false positive samples in the distribution.

16. The facial detection device according to claim 15, wherein, Within the size range of the detection box, the adaptive threshold decreases in response to a decrease in the size of the detection box.

17. The facial detection device according to claim 15, wherein, The processor is also configured to additionally set an adaptive threshold based on the quality of the input image.

18. The facial detection device according to claim 15, wherein, The processor is also configured to set an adaptive threshold by inputting the values ​​of the detection box size and the quality of the input image into the threshold setting engine, and The quality value is input as a weight into the threshold setting engine.

19. The facial detection device according to claim 15, wherein, The processor is also configured to generate an early detection result indicating whether an object in the detection box corresponds to a face by comparing the confidence score with at least one of a maximum threshold and a minimum threshold.

20. The facial detection device according to claim 19, wherein, The processor is also configured as follows: In response to a confidence score greater than the maximum threshold, an early detection result indicating that the object in the detection box corresponds to a face is generated. In response to a confidence score less than a minimum threshold, an early detection result indicating that the object in the detection box does not correspond to a face is generated. In response to a confidence score greater than the minimum threshold and less than the maximum threshold, the system determines whether an object in the detection box corresponds to a face by comparing the confidence score with an adaptive threshold.

21. The facial detection device according to claim 15, in, The processor is also configured to: calculate the quality of the input image and apply a compensation value based on the quality of the input image to the confidence score. The compensation value is set to increase the confidence score of low-quality images and / or decrease the confidence score of high-quality images.

22. The facial detection device according to claim 17, wherein, The compensation value is also based on the size of the detection box. The processor is configured to: in response to an increase in the size of the detection box, further increase the confidence score of low-quality images using compensation values, and / or further decrease the confidence score of high-quality images.

23. The facial detection device according to claim 15, further comprising: Memory is configured to store instructions that can be executed by the processor.

Images