IMAGE PROCESSING WITH FACE MASK DETECTION

Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed for automatically processing an image based on the detection of a face mask. An exemplary article of manufacture includes instructions that, when executed, cause programmable circuitry to at least: map a feature of an upper portion (402) of a person's face in an image to a color map including a facial skin tone cone (302); map a feature of a lower portion (404) of the face to the color map; and identify the presence or absence of a face mask based on the respective positions of the feature of the upper portion (402) and the feature of the lower portion (404) with respect to the skin tone cone (302).

Description

AREA OF REVELATION

[0001] This disclosure relates generally to image processing and more particularly to image processing with face mask detection. BACKGROUND

[0002] Image processing techniques such as automatic exposure and automatic white balance can be affected by a face mask, which typically has a different color and brightness than the facial skin. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram of an exemplary image processing system with exemplary face mask detection circuitry. Fig. 2 is a schematic diagram of an exemplary boundary around a facial area. Fig. Figure 3 is a graphical representation of an HSV (hue, saturation, brightness) color space versus an RGB (red, green, blue) color space, including an example skin tone cone. Fig. 4 is a schematic diagram of an exemplary boundary over the face area of Fig. 2. Fig. Figure 5 is a schematic diagram of the exemplary face of Fig. 2, which shows a rotation in the plane. Fig. 6 is a schematic diagram of the exemplary face of Fig. 2, which shows a rotation out of the plane. Fig. 7 is a flowchart illustrating exemplary machine-readable instructions and / or exemplary operations that may be performed by exemplary processor circuitry to implement the image processing system of Fig. 1 to be implemented. Fig. 8 is a block diagram of an example processing platform including processor circuitry structured to execute the example machine-readable instructions and / or the example operations of Fig. 7 to run the image processing system from Fig. 1 to be implemented. Fig. 9 is a block diagram of an exemplary implementation of the processor circuitry of Fig. 8. Fig. 10 is a block diagram of another exemplary implementation of the processor circuitry of Fig. 8. Fig. 11 is a block diagram of an example software distribution platform (e.g., one or more servers) for distributing software (e.g., software that conforms to the example machine-readable instructions of Fig. 7) to client devices associated with end users and / or consumers (e.g., for licensing, sale and / or use), retailers (e.g., for sale, resale, licensing and / or sublicensing) and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products for distribution, for example, to retailers and / or to other end users such as direct customers).

[0003] In general, the same reference numerals are used in the drawing(s) and the accompanying written description to refer to the same or similar parts. The figures are not to scale. DETAILED DESCRIPTION

[0004] Facial recognition techniques in images are used for various purposes, such as surveillance, security, personal identification, biometric login, permission to access a location, and / or electronic payment. Facial recognition is typically the first step for facial recognition and / or other related technologies. Facial recognition is also used in video conferencing, for example, where people work from home or study and attend classes from home.

[0005] Before images collected by a camera are processed for face detection, one or more aspects of the images can be adjusted. An example group of adjustments is known as 3A: automatic exposure (AE), automatic white balance (AWB), and automatic focus (AF). Automatic exposure operations adjust image brightness based on the amount of light reaching the camera sensor (exposure). Incorrect exposure can result in a washed-out or faded image (overexposure) or a dark image with little detail (underexposure). Automatic white balance operations compensate for color differences based on lighting, so that the color white in an image actually appears white. White objects can appear a different color in an image based on the color temperature of the light illuminating the object.For example, a white object illuminated by a low-color-temperature light source will have a reddish hue, while a white object illuminated by a high-color-temperature light source will have a bluish hue. Automatic focus operations use a sensor, control circuitry, and a motor to adjust the position of a lens to bring it into focus on a region of interest. Incorrect focus can cause the region of interest and / or an entire image to become blurred.

[0006] 3A adjustments can be specific to the faces of one or more people in images. Face-based 3A adjustment features are included in mobile phones, tablets, notebooks, and other electronic devices to enhance the user experience. With face-based 3A, the face is the area of interest, which has higher priority for image processing adjustments compared to the background and other areas of the image, regardless of backlighting, incident light, and / or other conditions captured in the image. Automatic face exposure is designed to ensure the correct brightness for the face. Automatic face white balance aims to preserve the natural skin color of the face rather than adjusting the color for the background. Automatic face focus uses the face for focus processing, ensuring that the face is the most prominent part of the scene in the image.However, if the user wears a face mask, brightness and / or color shifts of the entire image may occur after the 3A adjustments, since the brightness and color of a mask are usually significantly different from the brightness and color of a face.

[0007] Since the outbreak of the COVID-19 pandemic, face masks have become increasingly common in daily life. For example, individuals wearing face masks may attend meetings in person in a conference room that has a camera to film and / or stream the meeting for others attending the meeting remotely. However, when individuals wear a face mask, face-based automatic exposure and automatic white balance may be affected by a face mask because the face mask has a different color and brightness than the facial skin. According to the teachings of this disclosure, a face mask is automatically detected, and the face-based 3A adjustments are modified taking the presence of the face mask into account. In this way, color and brightness deviations of the image are avoided during face-based 3A adjustments.

[0008] The examples disclosed herein detect a face mask by comparing the difference in hue, chroma, and brightness averages between an upper half of a face and a lower half of a face in an image. If a face mask is detected, automatic exposure and automatic white balance are performed for the portion of the face not covered by a face mask. In the examples disclosed herein, when a face mask is detected, abnormal brightness and / or color shift caused by automatic exposure and / or automatic white balance on the face is avoided because the automatic exposure and / or automatic white balance do not adjust the skin of the face based on the color and / or brightness of a face mask.In this way, the skin of the person wearing a face mask appears without color and / or brightness abnormalities.

[0009] The examples disclosed herein include many advantages, such as accurately reproducing the color and brightness of facial skin of individuals wearing face masks during video streaming and / or video conferencing. The example face mask detection mechanisms disclosed herein are easily integrated into camera systems, such as smartphones, tablets, and computers with cameras. Furthermore, the examples disclosed herein perform image processing with lower power consumption compared to conventional face mask detection solutions and utilize strong neutral network / deep learning algorithms with complicated dataset collection, labeling, and training processes.

[0010] As used herein, an "image" includes still images, digital images, photographs, recorded video, live video, and / or streaming video. An image refers to any visual image captured by a camera, optical sensor, and / or other optical instrument.

[0011] Unless specifically stated otherwise, descriptors such as "first," "second," "third," etc., are used herein without in any way implying or otherwise indicating any meaning of priority, physical order, arrangement in a list, and / or order; rather, they are used merely as labels and / or arbitrary names to distinguish elements for easy understanding of the disclosed examples. In some examples, the descriptor "first" may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as "second" or "third." In these cases, it is understood that such descriptors are used merely to uniquely identify those elements, which might otherwise, for example, share the same label.

[0012] The term "in communication" as used herein, including variations thereof, includes direct communication and / or indirect communication through one or more intermediate components and does not require direct physical (e.g., wired) communication and / or constant communication, but additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or on one-time events.

[0013] The term “processor circuitry” as used herein is defined to include: (i) one or more special-purpose electrical circuits structured to perform one or more specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuits programmed with instructions to perform specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors).Examples of processor circuitry include programmable microprocessors, field-programmable gate arrays (FPGAs) that can instantiate instructions, central processor units (CPUs), graphics processor units (GPUs), digital signal processors (DSPs), XPUs or microcontrollers, and integrated circuits such as application-specific integrated circuits (ASICs). For example, an XPU may be implemented by a heterogeneous computing system that includes multiple types of processor circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more DSPs, etc.).and / or a combination thereof) and one or more application programming interfaces (APIs) that can assign one or more computational tasks to each of the plurality of types of processor circuitry best suited to performing the one or more computational tasks.

[0014] Fig. 1 is a block diagram of an exemplary image processing system 100 that can automatically detect a face mask and modify the image processing, including 3A adjustments, based on the detected face mask. The image processing system 100 includes exemplary image signal processor circuitry 102, exemplary face mask detection circuitry 104, and exemplary image adjustment circuitry 106. The face mask detection circuitry 104 includes, for example, exemplary face detection circuitry 108, exemplary color space conversion circuitry 110, exemplary skin tone cone mapping circuitry 112, exemplary color comparison circuitry 114, and exemplary calculation circuitry 116.The image adjustment circuitry includes, for example, an exemplary automatic exposure circuitry 118, an exemplary automatic white balance circuitry 120, and an exemplary automatic focus circuitry 122.

[0015] The image processing system 100 is in communication with an example camera 124 or other optical instrument or optical sensor. In some examples, the image processing system 100 is included within the camera 124. In some examples, the image processing system 100 and the camera 124 are included within the same electronic device. In some examples, the image processing system 100 is remote from the camera 124 and coupled via a wired or wireless connection.

[0016] The image processing system 100 accesses, receives, retrieves, or otherwise obtains images from the camera 124. The image signal processor circuitry 102 performs one or more example tasks. In some examples, the image signal processor circuitry 102 performs a Bayer transform by applying a Bayer filter to add red, green, and blue (RGB) colors to pixels recorded in grayscale by photodiodes of the camera 124. In some examples, the image signal processor circuitry 102 converts to the RGB color space using the YUV color model. The YUV model defines brightness or luminance (Y), blue projection (U), and red projection (V). In these examples, the image signal processor circuitry 102 uses one of several mapping formulas to convert the YUV data to RGB data.In some examples, image signal processor circuitry 102 interpolates missing color and brightness information using a demosaicing algorithm. In some examples, image signal processor circuitry 102 reduces noise in the images. Additionally, in some examples, image signal processor circuitry 102 also detects edges of objects in the images and sharpens the images. Image signal processor circuitry 102 outputs preprocessed images for further analysis by image processing system 100.

[0017] Face detection circuitry 108 accesses, receives, retrieves, or otherwise obtains the preprocessed images from image signal processor circuitry 102. In some examples, face detection circuitry 108 accesses, receives, retrieves, or otherwise obtains raw images directly from camera 124. In some examples, face detection circuitry 108 determines face detection data through one or more statistically based operations, such as scale-invariant feature transform (SIFT), local binary pattern (LBP), and / or the discriminatory latent variable Gaussian process model (also known as GaussianFace).In some examples, face detection circuitry 108 determines face detection data through a neural network, such as a deep neural network (DNN), a convolutional neural network (CNN), FaceNet, and / or multi-task cascaded neural networks (MTCNN).

[0018] The face detection circuitry 108 delimits a region of a face in an image. For example, the face detection circuitry 108 may draw a contour or boundary, such as a rectangle 200, around a face, as shown in Fig. 2. In other examples, the contour around the face may be elliptical, circular, and / or other shaped. Furthermore, face detection circuitry 108 may detect or extract features or landmarks of a face, such as eyes, nose, and mouth.

[0019] Color space conversion circuitry 110 converts the colors of the pixels within the contour 200 around the face from the RGB color space to the HSV color space. The HSV color space represents RGB colors the way human vision perceives color characteristics in terms of hue (H), chroma (S), and brightness (known as lightness) (V). In the HSV color space, hue, chroma, and lightness are separate and more distinguishable than the colors in the RGB color space. The distinguishability of hue, chroma, and lightness enables discrimination between pixels that are facial skin and pixels that are not facial skin (e.g., pixels of a face mask), according to teachings of this disclosure.

[0020] Typically, a face mask has a color that differs from the color or tone of the facial skin. For example, the face mask may have a bright color, such as light blue, light pink, bright white, or dark black. The RGB color space can be converted to the HSV color space using these example equations, for example, where C represents color: R'=R255 G'=G255 B'=B255 Cmax=ηmax(R', G', B') Cmin=min(R', G', B') Δ=Cmax−Cmin

[0021] Color calculation: H{0°>>060°∗(G'−R'Δ+2), Cmax=R'60°∗(B'−R'Δ+2), Cmax=G'60°∗(R'−G'Δ+4), Cmax=B'

[0022] Color saturation calculation: / S / {0, Cmax=0ΔCmax, Cmax≠0

[0023] Brightness value calculation: V=Cmax

[0024] The skin tone cone mapping circuitry 112 defines a skin tone cone. Fig. Figure 3 is a plot of HSV values against a range of RGB values, resulting in a cone of 300. Facial skin tone can vary from white to dark, but the hue value of facial skin tone cannot be a color such as green, cyan, and / or blue. Furthermore, the chroma of facial skin tone cannot encompass the entire chroma space. Thus, a facial skin tone can fall within the part of cone 300 defined by points A, B, C, and D in Fig. 3. The smaller area within points A, B, C, and D is a facial skin tone cone 302 (which is actually a wedge-shaped portion of cone 300). Facial skin tone cone 302 is calibrated based on known data and / or prior experimentation. Facial skin tone cone 302 is an estimate of complete collections of all possible facial skin tones, including various human races and ethnicities. In some examples, facial skin tone cone 302 may be configurable. The depicted facial skin tone cone 302 is used in the detection of a facial mask, as disclosed herein.

[0025] Face detection circuitry 108 analyzes the hue, chroma, and / or brightness / brightness values or values in contour 200 to determine whether there is a boundary line within contour 200. A boundary line could indicate a face mask. Fig. 4 is a schematic diagram of an exemplary boundary 400 across the facial region within contour 200. To obtain an accurate position of boundary line 400, calculation circuitry 116 may calculate the hue, chroma, and / or brightness / brightness averages row by row. If there is an abrupt change in hue, chroma, and / or brightness / brightness from one line to the next, boundary line 400 is detected, which may be an upper boundary of a face mask. Face detection circuitry 108 divides the region within facial contour 200 into a face / upper region 402 and a mask / lower region 404.

[0026] The face detection circuitry 108 can specify the position of the boundary line 400. For example, a person may tilt their head to one side, which corresponds to a rotation in plane (RIP), as shown in Fig. 5. The person can also turn his head to one side, which corresponds to a rotation out of plane (ROP), as in Fig. 6. The face detection circuitry 108 may determine a more precise and / or exact mask position by taking into account the degree of RIP and / or ROP as a person moves.

[0027] Furthermore, the face detection circuitry 108 can determine the coordinates of the eyes 500. The face detection circuitry 108 identifies the upper boundary of the face mask, i.e., the boundary line 400, which is located below the eyes 500.

[0028] If the boundary line 400 is not detected, the face detection circuitry 108 divides the area within the contour 200 into two equal halves to form the upper area 402 and the lower area 404.

[0029] For the upper region 402, the calculation circuitry 116 calculates the average values of hue (Hu), chroma (Su), and brightness (Vu). In addition, for the lower region 404, the calculation circuitry 116 calculates the average values of hue (HI), chroma (SI), and brightness (VI). The calculation circuitry 116 determines the combination of the averages of these metrics as Pu (Hu, Su, Vu) and PI (HI, SI, VI), respectively, to represent the average points of the upper-half region 402 and the lower-half region 404, respectively.

[0030] Skin tone cone mapping circuitry 112 maps the upper region mean Pu and the lower region mean PI to the cone 300. Color comparison circuitry 114 compares the upper region mean Pu and the lower region mean PI to the facial skin tone cone 302. There are four combinations for the position of the upper region mean Pu and the lower region mean Pl with respect to the facial skin tone cone 302: (1) both Pu and PI lie within the facial skin tone cone 302; (2) Pu lies within the facial skin tone cone 302 and PI lies outside the facial skin tone cone 302; (3) Pu lies outside the facial skin tone cone 302 and PI lies within the facial skin tone cone 302; and (4) both Pu and PI lie outside the facial skin tone cone 302.The second combination, where Pu is within the facial skin tone cone 302 and PI is outside the facial skin tone cone 302, is shown in . Fig. 3 shown.

[0031] For the first combination, where both Pu and PI lie within the facial skin tone cone 302, further evaluation is required to determine whether a face mask is present. For further evaluation, the calculation circuitry 116 calculates the distance between the upper region mean Pu and the lower region mean Pl. In some examples, the distance may be calculated as follows: Distance(Hl−Hu)2+(Sl−Su)2+(Vl−Vu)2

[0032] The distance is the difference between the upper region 402 of the face and the lower region 404 of the face. The color comparison circuitry 114 determines whether the distance is greater than a distance threshold. The threshold may be established through prior studies and / or experiments with subjects known to wear face masks. If the color comparison circuitry 114 determines that the distance is greater than the distance threshold, the face mask detection circuitry 104 determines that the person is wearing a face mask. If the color comparison circuitry 114 determines that the distance is not greater than the distance threshold, the face mask detection circuitry 104 determines that the person is not wearing a face mask.

[0033] Because face masks typically have a different color and / or brightness than facial skin, in other examples, calculation circuitry 116 determines the distance between the upper portion 402 of the face and the lower portion 404 of the face by comparing one of the hue, chroma, and / or brightness / brightness averages between the upper portion 402 of the face and the lower portion 404 of the face. Calculation circuitry 116 determines the absolute value of the respective differences in the hue, chroma, and / or brightness / brightness averages between the upper portion 402 of the face and the lower portion 404 of the face. Thus, calculation circuitry 116 may alternatively calculate the distance between the upper portion 402 of the face and the lower portion 404 of the face using one of the following equations: Distance(Hu−Hl)| Distance(Su−Sl)| Distance(Vu−Vl)|

[0034] Calculation circuitry 116 compares these absolute values to respective thresholds for hue, chroma, and brightness / brightness. The thresholds may be established through prior studies and / or trials with subjects known to be wearing face masks. |(Hu−Hl)|>THh |(Su−Sl)|>THs |(Vu−Vl)|>THv

[0035] If the absolute value of the difference for the hue, the color saturation and / or the brightness / brightness value is greater than the respective threshold value TH h , TH s or TH v , the face mask detection circuitry 104 determines that the person is wearing a face mask. If the absolute value of the difference for either the hue or the color saturation or the brightness / brightness is not greater than the respective threshold value TH h , THs or TH v , the face mask detection circuitry 104 determines that the person is not wearing a face mask.

[0036] In some examples, it is possible for a person to wear a face mask and both Pu and Pi to be within the facial skin tone cone 302 because the face mask may have a skin tone color. For example, a person with light skin may wear a black mask, or a person with dark skin may wear a white mask.

[0037] For the second combination mentioned above, where Pu is within facial skin tone cone 302 and PI is outside facial skin tone cone 302, face mask detection circuitry 104 determines that the subject is likely wearing a face mask. Face mask detection circuitry 104 verifies this conclusion by calculating the distance between upper region 402 and lower region 404 using one or more examples disclosed herein. If the distance is greater than the threshold, face mask detection circuitry 104 confirms that the subject is wearing a mask. If the distance is less than the threshold, Pu and Pl are too close together, and face mask detection circuitry 104 does not determine that the subject is wearing a mask.

[0038] For the third combination, where Pu is outside the facial skin tone cone 302 and PI is within the facial skin tone cone 302, the face mask detection circuitry 104 determines that the person is likely not wearing a face mask. The person may be wearing a hat, other headgear, large ski goggles, etc. The face mask detection circuitry 104 verifies this conclusion by calculating the distance between the upper region 402 and the lower region 404 using one or more examples disclosed herein. If the distance is greater than the threshold, the face mask detection circuitry 104 confirms that the person is wearing a hat or other object on the top of their head. If the distance is less than the threshold, Pu and PI are too close together, and the face mask detection circuitry 104 makes no determination about what the person is wearing.

[0039] For the fourth combination, where both Pu and PI lie outside the facial skin tone cone 302, the facial detection circuitry 104 determines that the results are inconclusive. The facial data within the contour 200 may be unreliable. For example, the facial data may be unreliable because the person is moving, there is a bright background, the facial data is not updated in a timely manner, etc.

[0040] After determining the information about the presence or absence of a face mask, image adjustment circuitry 106 modifies the image. Automatic focus circuitry focuses the image on the area within contour 200, regardless of whether a face mask is present or not. The operations of automatic exposure circuitry 118 and automatic white balance circuitry 120 are adjusted.

[0041] If face mask detection circuitry 108 determines that the subject is wearing a face mask, the exposed facial area (upper area 402) is processed by automatic exposure circuitry 118 and automatic white balance circuitry 120 in accordance with the automatic exposure and automatic white balance protocols. The face mask area (lower area 404) is not processed by automatic exposure circuitry 118 and automatic white balance circuitry 120.The lower region 404 is not subjected to automatic exposure processing if a face mask is worn by the subject, since processing the face mask region with automatic exposure results in an image that is too dark when the face mask is lighter than the facial skin, and processing the face mask region with automatic exposure results in an image that is too bright when the face mask is less bright than the facial skin. Images that are too dark or too bright are unacceptable and not expected by users (e.g., people in a video conference). Furthermore, the lower region 404 is not subjected to automatic white balancing if the subject is wearing a face mask, since processing the face mask region with automatic white balancing results in a color shift, which is also not acceptable or expected.In some examples, for individuals with thick and / or dark beards, the beard area is omitted from automatic exposure or automatic white balance processing in the same way and for the same reasons as when a face mask is present.

[0042] If the face mask detection circuitry 108 determines that the person is not wearing a face mask, the entire area in the contour 200 (both the upper area 402 and the lower area 404) is processed by the automatic exposure circuitry 118 and the automatic white balance circuitry 120 in accordance with the automatic exposure and automatic white balance protocols.

[0043] Fig. 1 is a block diagram of the image processing system 100 for processing images based on face mask detection. The image processing system 100 of Fig. 1 may be instantiated (e.g., create an instance, realize it for an arbitrary period of time, materialize it, implement it, etc.) by processor circuitry such as a central processing unit that executes instructions. Additionally or alternatively, the image processing system 100 may be comprised of Fig. 1 be instantiated (e.g., create an instance, realize it for an arbitrary period of time, materialize it, implement it, etc.) by an ASIC or an FPGA that is structured to perform operations that correspond to the instructions. It is understood that part or all of the circuitry consists of Fig. 1 can thus be instantiated simultaneously or at different times. Part or all of the circuitry can, for example, be instantiated in one or more threads that execute concurrently on hardware and / or in series on hardware. Furthermore, the circuitry can consist of Fig. 1 in some examples may be implemented in part or in whole by microprocessor circuitry that executes instructions to implement one or more virtual machines and / or containers.

[0044] In some examples, the face detection circuitry 108 of the face mask detection circuitry 104 is instantiated by processor circuitry that executes face detection instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7. In some examples, the color space conversion circuitry 110 of the face mask detection circuitry 104 is instantiated by processor circuitry that executes color space conversion instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7. In some examples, the skin tone cone mapping circuitry 112 of the face mask detection circuitry 104 is instantiated by processor circuitry that executes skin tone cone mapping instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7. In some examples, the color comparison circuitry 114 of the face mask detection circuitry 104 is instantiated by processor circuitry that executes color comparison instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7. In some examples, the computation circuitry (116) of the face mask detection circuitry 104 is instantiated by processor circuitry that executes computation instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7 shown.

[0045] In some examples, the automatic exposure circuitry 118 of the face mask detection circuitry 106 is instantiated by processor circuitry that executes automatic exposure instructions and / or is configured to perform operations such as those shown in the flowchart of Fig. 7. In some examples, the automatic white balance circuitry 120 of the face mask detection circuitry 106 is instantiated by processor circuitry that executes automatic white balance instructions and / or is configured to perform operations such as those illustrated in the flowchart of Fig. 7. In some examples, the automatic focus circuitry 122 of the face mask detection circuitry 106 is instantiated by processor circuitry that executes automatic focus instructions and / or is configured to perform operations such as those shown in the flowchart of Fig. 7 shown.

[0046] In some examples, the device includes means for detecting a face mask. The means for detecting may be implemented, for example, by face mask detection circuitry 104. In some examples, face mask detection circuitry 104 may be implemented by processor circuitry, such as the example processor circuitry 812 of Fig. 8. The face mask detection circuitry 104 may be implemented, for example, by the exemplary microprocessor 900 of Fig. 9, which executes machine-executable instructions as described at least in blocks 702 - 728 and 732 of Fig. 7. In some examples, the face mask detection circuitry 104 may be instantiated by hardware logic circuitry implemented by an ASIC, an XPU, or the FPGA circuitry 1000 of Fig. 10 and is structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the face mask detection circuitry 104 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the face mask detection circuitry 104 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier (op-amp), logic circuitry, etc.) structured to perform some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other structures are also suitable.

[0047] In some examples, the device includes means for adjusting an image. The means for adjusting may be implemented, for example, by image adjustment circuitry 106. In some examples, image adjustment circuitry 106 may be implemented by processor circuitry, such as example processor circuitry 812 of Fig. 8. The image adjustment circuitry 106 may be instantiated, for example, by the exemplary microprocessor 900 of Fig. 9, which executes machine-executable instructions as defined at least by blocks 730 and 734 of Fig. 7. In some examples, the image adjustment circuitry 106 may be instantiated by hardware logic circuitry implemented by an ASIC, an XPU, or the FPGA circuitry 1000 of Fig. 10 and is structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the image adjustment circuitry 106 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the face mask detection circuitry 106 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier (op-amp), logic circuitry, etc.) structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, although other structures are also suitable.

[0048] Although in Fig. 1 illustrates an exemplary manner of implementing the image processing system 100, one or more of the Fig. 1 may be combined, divided, rearranged, omitted, eliminated, and / or implemented in a different manner. Furthermore, the exemplary image signal processor circuitry 102, the exemplary face mask detection circuitry 104, the exemplary image adjustment circuitry 106, the exemplary face detection circuitry 108, the exemplary color space conversion circuitry 110, the exemplary skin tone cone mapping circuitry 112, the exemplary color comparison circuitry 114, the exemplary calculation circuitry 116, the exemplary automatic exposure circuitry 118, the exemplary automatic white balance circuitry 120, the exemplary automatic focus circuitry 122, and / or more generally, the exemplary image processing system 100 may be Fig. 1 may be implemented by hardware alone or by hardware in combination with software and / or firmware. Thus, for example, any of the example image signal processor circuitry 102, the example face mask detection circuitry 104, the example image adjustment circuitry 106, the example face detection circuitry 108, the example color space conversion circuitry 110, the example skin tone cone mapping circuitry 112, the example color comparison circuitry 114, the example calculation circuitry 116, the example automatic exposure circuitry 118, the example automatic white balance circuitry 120, the example automatic focus circuitry 122, and / or more generally, the example image processing system 100 may be implemented by processor circuitry,one or more analog circuits, one or more digital circuits, one or more logic circuits, one or more programmable processors, one or more programmable microcontrollers, one or more graphics processing units (GPU(s)), one or more digital signal processors (DSP(s)), one or more application-specific integrated circuits (ASIC(s)), one or more programmable logic devices (PLD(s)), and / or one or more field-programmable logic devices (FPLD(s)) such as field-programmable gate arrays (FPGAs). Furthermore, the exemplary image processing system 100 of FIG. Fig. 1 one or more elements, processes and / or devices in addition to those described in Fig. 1 or in place of those shown and / or include more than one or all of the elements, processes and devices shown.

[0049] A flowchart illustrating exemplary machine-readable instructions that may be executed to configure processor circuitry to implement the image processing system 100 of Fig. 1 is to be implemented in Fig. 7. The machine-readable instructions may be one or more executable programs or one or more portions of an executable program for execution by processor circuitry, such as processor circuitry 812 described in the application described below in connection with Fig. 8, and / or described below in connection with the Fig. 9 and / or 10. The program may be embodied in software stored on one or more non-transitory computer-readable storage media, such as a compact disk (CD), a floppy disk, a hard disk drive (HDD), a solid state drive (SSD), a digital versatile disk (DVD), a Blu-ray disk, volatile memory (e.g., random access memory (RAM) of any type, etc.), or non-volatile memory (e.g., electrically erasable programmable read-only memory (EEPROM), flash memory, an HDD, an SSD, etc.).) associated with processor circuitry located in one or more hardware devices; alternatively, the entire program and / or portions thereof may be executed by one or more hardware devices other than the processor circuitry and / or embodied in firmware or dedicated hardware. The machine-readable instructions may be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). The client hardware device may, for example, be an endpoint client hardware device (e.g., a hardware device associated with a user) or an intermediate client hardware device (e.g.,a radio access network (RAN) gateway that can enable communication between a server and an endpoint client hardware device). Similarly, the non-transitory computer-readable storage media can include one or more media located in one or more hardware devices. Although the example program is described with reference to the embodiments described in . Fig. 7, numerous other methods for implementing the example image processing system 100 may alternatively be used. For example, the execution order of the blocks may be changed, and / or some of the described blocks may be changed, omitted, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, an FPGA, an ASIC, a comparator, an operational amplifier (op-amp), logic circuitry, etc.) structured to perform the corresponding operation without executing software or firmware. The processor circuitry may be deployed to various network locations and / or locally to one or more hardware devices (e.g., a single-core processor (e.g.,a single-core central processor unit (CPU), a multi-core processor (e.g., a multi-core CPU), etc.) in a single machine, multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server racks, a CPU and / or an FPGA located in the same package (e.g., the same integrated circuit (IC) package or in two or more separate packages, etc.).

[0050] The machine-readable instructions described herein may be stored in a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, and / or a packaged format, etc. Machine-readable instructions as described herein may be stored as data or a data structure (e.g., portions of instructions, code, representations of code, etc.) that are usable to generate, manufacture, and / or produce machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored on one or more storage and / or computing devices (e.g., servers) located at the same or different locations of a network or collection of networks (e.g., in the cloud, at edge devices, etc.).The machine-readable instructions may require one or more of the following operations: installing, modifying, adapting, updating, combining, supplementing, configuring, decrypting, decompressing, unpacking, distributing, remapping, compiling, etc., to make them directly readable, interpretable, and / or executable by a computing device and / or another machine. For example, the machine-readable instructions may be stored in multiple pieces that are individually compressed, encrypted, and / or stored on separate computing devices, which pieces, when decrypted, decompressed, and / or combined, form a set of machine-executable instructions that implement one or more operations that together may form a program such as the one described herein.

[0051] In another example, the machine-readable instructions may be stored in a state in which they can be read by processor circuitry, but which may require the addition of a library (e.g., a dynamic link library (DLL), a software development kit (SDK), an application programming interface (API), etc.) to execute the machine-readable instructions on a particular computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., settings saved, data entered, network addresses recorded, etc.) before the machine-readable instructions and / or the one or more corresponding programs can be executed in whole or in part.Thus, machine-readable media, as used herein, may contain machine-readable instructions and / or one or more programs regardless of the particular format or state of the machine-readable instructions and / or one or more programs when stored or otherwise at rest or in transition.

[0052] The machine-readable instructions described herein may be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, the machine-readable instructions may be represented by any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0053] As mentioned above, the example operations from Fig. 7 may be implemented using executable instructions (e.g., computer- and / or machine-readable instructions) stored on one or more non-transitory computer- and / or machine-readable media, such as optical storage devices, magnetic storage devices, an HDD, flash memory, read-only memory (ROM), a CD, a DVD, a cache, RAM of any type, a register, and / or any other storage device or disk on which information is stored for any duration (e.g., for extended periods of time, permanently, for short periods of time, for temporarily buffering and / or caching the information).As used herein, the terms "non-transitory computer-readable medium," "non-transitory computer-readable storage medium," "non-transitory machine-readable medium," and / or "non-transitory machine-readable storage medium" are expressly defined to include any type of computer-readable storage device and / or storage disk, and exclude propagating signals and transmission media. As used herein, the terms "computer-readable storage device" and "machine-readable storage device" are defined to include any physical (mechanical and / or electrical) structure for storing information, and exclude propagating signals and transmission media.Examples of non-transitory computer-readable storage devices and / or machine-readable storage devices include random access memory (RAM) of any type, read-only memory (ROM) of any type, solid-state memory, flash memory, optical disks, magnetic disks, disk drives, and / or RAID (Redundant Array of Independent Disks) systems. The term "device," as used herein, refers to a physical structure, such as mechanical and / or electrical equipment, hardware, and / or circuitry, that may or may not be configured by computer-readable instructions, machine-readable instructions, etc., and / or may or may not be fabricated to execute computer-readable instructions, machine-readable instructions, etc.

[0054] "Including" and "comprising" (and all forms and tenses thereof) are used herein as open-ended terms. Thus, when any form of "include" or "comprise" (e.g., comprises, includes, comprising, including, having, etc.) is used in a claim as a preamble or in any claim language, it is understood that additional elements, labels, etc. may be present without departing from the scope of the corresponding claim or language. When the phrase "at least" ("at least"), as used herein, is used as a transitional term in, for example, a preamble to a claim, it is an open-ended term in the same way that the terms "comprising" and "including" are open-ended terms.The term “and / or,” when used, for example, in a form such as A, B, and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, or (7) A with B and with C. The phrase “at least one of A and B,” as used herein in connection with the description of structures, components, items, objects, and / or things, is intended to refer to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Likewise, the phrase “at least one of A or B,” as used herein in connection with the description of structures, components, items, objects, and / or things, is intended to refer to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.The phrase “at least one of A and B,” as used herein in connection with describing the performance or execution of processes, instructions, acts, activities, and / or steps, is intended to refer to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Likewise, the phrase “at least one of A or B,” as used herein in connection with describing the performance or execution of processes, instructions, acts, activities, and / or steps, is intended to refer to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0055] References in the singular (e.g., "a," "an," "first," "second," etc.) as used herein do not exclude plurality. The term "an" object, as used herein, refers to one or more of such objects. The terms "a," "one or more," and "at least one" are used interchangeably herein. Further, multiple means, elements, or method acts may be implemented by the same entity or object, among others, even if listed individually. Furthermore, individual features that may be included in different examples or claims may also be combinable, and inclusion in different examples or claims does not imply that a combination of features is not possible and / or advantageous.

[0056] Fig. 7 is a flowchart representative of exemplary machine-readable instructions and / or exemplary operations 700 that may be executed and / or instantiated by processor circuitry to detect a face mask in an image. The machine-readable instructions and / or operations 700 of Fig. 7 include face detection circuitry 108 accessing face detection data (block 702). Face detection circuitry 108 delimits a region of the face (block 704).

[0057] The color space conversion circuitry 110 converts the color of the face (RGB colors of the pixels of the face) within the contour area to the HSV color space (block 706). The skin tone cone mapping circuitry 112 maps a facial skin tone cone, as in the Fig. 3 (block 708).

[0058] Face detection circuitry 108 determines whether there is a boundary line across the area of the face within the contour (block 710). If there is no boundary line (block 710: NO), face detection circuitry 108 equally divides the area of the face within the contour into an upper region and a lower region (block 712). If there is a boundary line (block 710: YES), face detection circuitry 108 identifies the upper region and lower region of the face within the contour based on the boundary line (block 714). In some examples, face detection circuitry 108 confirms and / or refines the position of the boundary line using the eye coordinate position, the RIP degree data, and / or the ROP degree data, as disclosed herein.

[0059] After face detection circuitry 108 generates and / or identifies the upper and lower regions of the face, calculation circuitry 116 determines the hue, chroma, and brightness / lightness averages across multiple pixels in the upper and lower regions of the face and aggregates or combines the metrics into an upper HSV combination (Pu) and a lower HSV combination (PI) (block 716).

[0060] Skin tone cone mapping circuitry 112 maps Pu and Pl to the skin tone cone (block 718). Color comparison circuitry 114 compares the respective positions of Pu and Pl to the skin tone cone to determine whether at least Pu and / or Pl lie within the skin tone cone (block 720). If neither Pu nor Pl lie within the skin tone cone (block 720: NO), face mask detection circuitry 104 determines that the results are inconclusive (block 722). In other words, face mask detection circuitry 104 does not determine whether a person in the image is wearing a face mask or not.

[0061] If Pu and / or Pl are within the skin tone cone (block 720: YES), a face mask may be present in the image, and the face mask detection circuitry 104 verifies the result by calculating the distance. Specifically, in some examples, the calculation circuitry 116 calculates the distance between the upper region and the lower region (block 724). The distance between the upper region and the lower region is a comparison of the hue, chroma, and / or brightness / brightness of the upper region and the lower region. The distance may be calculated, for example, using one or more of the equations disclosed above.

[0062] Face detection circuitry 108 determines whether the distance is greater than a threshold (block 726). If the distance is not greater than the threshold (block 726: NO), face detection circuitry 108 determines that the person in the image is not wearing a face mask (block 728). If the person is not wearing a face mask, image adjustment circuitry 106 processes the entire face area of the image. For example, automatic exposure circuitry 118 adjusts the exposure of the entire face area, and automatic white balance circuitry 120 adjusts the white balance of the entire face area (block 730).

[0063] If the distance is greater than the threshold (block 726: YES), face detection circuitry 108 determines that the person in the image is wearing a face mask (block 732). If the person is wearing a face mask, image adjustment circuitry 106 processes only the upper portion of the face in the image. For example, automatic exposure circuitry 118 adjusts the exposure of the upper face region, automatic white balance circuitry 120 adjusts the white balance of the upper face region, and the lower face region is skipped (block 734).The automatic exposure circuitry 118 and the automatic white balance circuitry 120 process the upper region and not the lower region, so that the skin of the upper region is adjusted appropriately and the colors and brightness are not distorted by the automatic exposure and automatic white balance of the face mask.

[0064] Fig. 8 is a block diagram of an exemplary processor platform 800 structured to execute the machine-readable instructions and / or operations of Fig. 7 to execute and / or instantiate the image processing system 100 from Fig. 1. The processor platform 800 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as an iPad™), a personal digital assistant (PDA), an internet device, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, a headset (e.g., an augmented reality (AR) headset, a virtual reality (VR) headset, etc.), or another portable device or any other type of computing device.

[0065] The processor platform 800 of the illustrated example includes processor circuitry 812. The processor circuitry 812 of the illustrated example is hardware. The processor circuitry 812 may be implemented, for example, by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers of any desired family or manufacturer. The processor circuitry 812 may be implemented by one or more semiconductor-based (e.g., silicon-based) devices.In this example, processor circuitry 812 implements example image processing system 100, example image signal processor circuitry 102, example face mask detection circuitry 104, example image adjustment circuitry 106, example face detection circuitry 108, example color space conversion circuitry 110, example skin tone cone mapping circuitry 112, example color comparison circuitry 114, example calculation circuitry 116, example auto-exposure circuitry 118, example auto-white balance circuitry 120, and example auto-focus circuitry 122.

[0066] The processor circuitry 812 of the illustrated example includes a local memory 813 (e.g., a cache, registers, etc.). The processor circuitry 812 of the illustrated example is in communication via a bus 818 with a main memory that includes a volatile memory 814 and a non-volatile memory 816. The volatile memory 814 can be implemented by SDRAM (Synchronous Dynamic Random Access Memory), DRAM (Dynamic Random Access Memory), RDRAM ® (RAMBUS ® Dynamic Random Access Memory (DRAM) and / or any other type of RAM device. The non-volatile memory 816 may be implemented by flash memory and / or any other desired type of storage device. Access to the main memory 814, 816 of the illustrated example is controlled by a memory controller 817.

[0067] The processor platform 800 of the illustrated example also includes an interface circuitry 820. The interface circuitry 820 may be implemented by hardware according to any type of interface standard, such as an Ethernet interface, a USB (Universal Serial Bus) interface, a Bluetooth ® interface, an NFC (near field communication) interface, a Peripheral Component Interconnect (PCI) interface and / or a Peripheral Component Interconnect Express (PCIe) interface.

[0068] In the illustrated example, one or more input devices 822 are connected to the interface circuitry 820. The one or more input devices 822 enable a user to input data and / or commands into the processor circuitry 812. The one or more input devices 822 may be implemented, for example, by an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, an isopoint, and / or a speech recognition system.

[0069] Also connected to the interface circuitry 820 of the illustrated example is one or more output devices 824. The one or more output devices 824 may be implemented, for example, by display devices (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT), an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuitry 820 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics processor circuit such as a GPU.

[0070] The interface circuitry 820 of the illustrated example also includes a communication device, such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and / or a network interface, to enable data exchange with external machines (e.g., computing devices of any type) over a network 826. Communication may occur, for example, via an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a wireless line-of-sight system, a cellular telephone system, an optical connection, etc.

[0071] The processor platform 800 of the illustrated example also includes one or more mass storage devices 828 for storing software and / or data. Examples of such mass storage devices 828 include magnetic storage devices, optical storage devices, floppy disk drives, HDDs, CDs, Blu-ray disk drives, redundant array of independent disks (RAID) systems, solid-state storage devices such as flash memory devices and / or SSDs, and DVD drives.

[0072] The machine-readable instructions 832 replaced by the machine-readable instructions from Fig. 7 may be stored in the mass storage device 828, in the volatile memory 814, in the non-volatile memory 816, and / or on a removable non-volatile computer-readable storage medium such as a CD or DVD.

[0073] Fig. 9 is a block diagram of an exemplary implementation of the processor circuitry 812 of Fig. 8. In this example, the processor circuitry 812 is Fig. 8 is implemented by a microprocessor 900. The microprocessor 900 may be, for example, a general-purpose microprocessor (e.g., a general-purpose microprocessor circuitry). The microprocessor 900 executes some or all of the machine-readable instructions of the flowchart. Fig. 7 to display the circuit arrangement Fig. 1 as logic circuits for performing the operations corresponding to these machine-readable instructions. In some of these examples, the circuitry is Fig. 1 is instantiated by the hardware circuitry of microprocessor 900 in combination with the instructions. Microprocessor 900 may be implemented, for example, by multi-core hardware circuitry such as a CPU, a DSP, a GPU, an XPU, etc. The microprocessor 900 of this example is a multi-core semiconductor device including N cores, although it may include any number of example cores 902 (e.g., 1 core). The cores 902 of microprocessor 900 may operate independently or cooperate to execute machine-readable instructions. Machine code corresponding to a firmware program, an embedded software program, or a software program may, for example, be executed by one of the cores 902, or it may be executed by multiple cores 902 at the same time or at different times.In some examples, the machine code corresponding to the firmware program, the embedded software program, or the software program is divided into threads and executed in parallel by two or more of the cores 902. The software program may correspond to some or all of the machine-readable instructions and / or operations represented by the flowchart of FIG. Fig. 7 are shown.

[0074] The cores 902 may communicate via a first example bus 904. In some examples, the first bus 904 may be implemented by a communication bus to facilitate communication associated with one or more of the cores 902. The first bus 904 may be implemented, for example, by at least one of an I2C (Inter-Integrated Circuit) bus, a SPI (Serial Peripheral Interface) bus, a PCI bus, and / or a PCIe bus. Additionally or alternatively, the first bus 904 may be implemented by any other type of compute bus or electrical bus. The cores 902 may receive data, instructions, and / or signals from one or more external devices via example interface circuitry 906. The cores 902 may output data, instructions, and / or signals to one or more external devices via the interface circuitry 906.Although the cores 902 of this example include an example local memory 920 (e.g., Level 1 (L1) cache, which may be divided into an L1 data cache and an L1 instruction cache), the microprocessor 900 also includes an example shared memory 910 that may be shared between the cores (e.g., Level 2 (L2) cache) to enable fast access to data and / or instructions. Data and / or instructions may be transferred (e.g., shared) by writing to and / or reading from the shared memory 910. The local memory 920 of each of the cores 902 and the shared memory 910 may be part of a hierarchy of data storage devices and may include multiple levels of cache and main memory (e.g., main memory 814, 816 of FIG. Fig. 8). Typically, higher storage levels in the hierarchy have shorter access times and smaller storage capacities than lower storage levels. Changes at the different levels of the cache hierarchy are managed (e.g., coordinated) by a cache coherence policy.

[0075] Each core 902 may be referred to as a CPU, DSP, GPU, etc., or any other type of hardware circuitry. Each core 902 includes control unit circuitry 914, arithmetic and logic (AL) circuitry (also referred to as ALU) 916, a plurality of registers 918, local memory 920, and a second example bus 922. Other structures may also be present. For example, each core 902 may include vector unit circuitry, single-instruction-multiple-data (SIMD) unit circuitry, load / store unit (LSU) circuitry, branch / jump unit circuitry, floating-point unit (FPU) circuitry, etc. The control unit circuitry 914 includes semiconductor-based circuits structured to control (e.g., coordinate) data movements within the corresponding core 902.The AL circuitry 916 includes semiconductor-based circuitry structured to perform one or more mathematical and / or logical operations on the data within the corresponding core 902. The AL circuitry 916 of some examples performs integer-based operations. In other example cases, the AL circuitry 916 also performs floating-point operations. In still other examples, the AL circuitry 916 may include first AL circuitry that performs integer-based operations and second AL circuitry that performs floating-point operations. In some examples, the AL circuitry 916 may be referred to as an arithmetic logic unit (ALU).Registers 918 are semiconductor-based structures for storing data and / or instructions, such as results of one or more of the operations performed by the AL circuitry 916 of the corresponding core 902. Registers 918 may include, for example, vector registers, SIMD registers, general-purpose registers, flag registers, segment registers, machine-specific registers, instruction pointer registers, control registers, debug registers, memory management registers, machine check registers, etc. Registers 918 may be arranged in a bank, as shown in FIG. Fig. 9. Alternatively, the registers 918 may be organized in any other arrangement, format, or structure, including distribution within the core 902 to reduce access time. The second bus 922 may be implemented by at least one of an I2C bus, an SPI bus, a PCI bus, and / or a PCIe bus.

[0076] Each core 902 and / or more generally, the microprocessor 900 may include additional and / or alternative structures to those shown and described above. For example, one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more converged / common mesh stops (CMSs), one or more shifters (e.g., barrel shifters), and / or other circuitry may be present. The microprocessor 900 is a semiconductor device fabricated to include many interconnected transistors to implement the structures described above in one or more integrated circuits (ICs) included in one or more packages. The processor circuitry may include and / or cooperate with one or more accelerators.In some examples, accelerators are implemented by logic circuitry to perform certain tasks faster and / or more efficiently than is possible with a general-purpose microprocessor. Examples of accelerators include ASICs and FPGAs, such as those discussed herein. A GPU or other programmable device may also be an accelerator. Accelerators may be located within the processor circuitry, in the same chip package as the processor circuitry, and / or in one or more packages separate from the processor circuitry.

[0077] Fig. 10 is a block diagram of another exemplary implementation of the processor circuitry 812 of Fig. 8. In this example, the processor circuitry 812 is implemented by the FPGA circuitry 1000. The FPGA circuitry 1000 may, for example, be implemented by an FPGA. The FPGA circuitry 1000 may, for example, be used to perform operations that would otherwise be performed by the exemplary microprocessor 900 of Fig. 5, which executes corresponding machine-readable instructions. However, once configured, the FPGA circuitry 1000 instantiates the machine-readable instructions in hardware and can thus often perform the operations faster than would be possible with a general-purpose microprocessor executing the corresponding software.

[0078] In contrast to the microprocessor 900 described above from Fig. 9 (which is a general-purpose device that may be programmed to carry out some or all of the machine-readable instructions represented by the flowchart of Fig. 7, whose connections and logic circuit are fixed after production) contains the FPGA circuit arrangement 1000 of the example from Fig. 10 in particular interconnections and logic circuitry which can be configured and / or connected to one another in different ways after manufacture, for example to implement some or all of the functions illustrated by the flowchart of Fig. 7. In particular, the FPGA circuitry 1000 can be viewed as an array of logic gates, interconnects, and switches. The switches can be programmed to change the way the logic gates are connected to one another via the interconnects, thereby forming one or more dedicated logic circuits (unless and until the FPGA circuitry 1000 is reprogrammed). The configured logic circuits enable the logic gates to interact in different ways to perform various operations on the data received by the input circuitry. These operations may correspond to some or all of the software represented by the flowchart of Fig. 7. The FPGA circuitry 1000 may be structured to execute some or all of the machine-readable instructions of the flowchart of Fig. 7 as dedicated logic circuits to perform the operations corresponding to these software instructions in a dedicated manner analogous to an ASIC. Thus, the FPGA circuit arrangement 1000 can perform the operations corresponding to part or all of the machine-readable instructions from Fig. 7 faster than the general-purpose microprocessor can perform them.

[0079] In the example from Fig. 10, the FPGA circuitry 1000 is structured to be programmed (and / or reprogrammed one or more times) by an end user via a hardware description language (HDL) such as Verilog. The FPGA circuitry 1000 of Fig. 6 includes example input / output (I / O) circuitry 1002 for receiving and / or outputting data to example configuration circuitry 1004 and / or external hardware 1006. For example, configuration circuitry 1004 may implement interface circuitry that may receive machine-readable instructions for configuring FPGA circuitry 1000 or one or more portions thereof. In some of these examples, configuration circuitry 1004 may receive the machine-readable instructions from a user, a machine (e.g., hardware circuitry (e.g., programmed or dedicated circuitry) that may implement an artificial intelligence / machine learning (AI / ML) model to generate the instructions), etc. In some examples, external hardware 1006 may be implemented by external hardware circuitry.The external hardware 1006 can be implemented, for example, by the microprocessor 900 of FIG. Fig. 9. The FPGA circuitry 1000 also includes an array of exemplary logic gate circuitry 1008, a plurality of exemplary configurable interconnects 1010, and exemplary memory circuitry 1012. The logic gate circuitry 1008 and the configurable interconnects 1010 are configurable to instantiate one or more operations corresponding to at least some of the machine-readable instructions of Fig. 7 and / or other desired operations. The Fig. The logic gate circuitry 1008 shown in Figure 10 is fabricated in groups or blocks. Each block includes semiconductor-based electrical structures that are configurable into logic circuits. In some examples, the electrical structures include logic gates (e.g., AND gates, OR gates, NOR gates, etc.) that provide basic building blocks for logic circuits. Electrically controllable switches (e.g., transistors) are present within each logic gate circuitry 1008 to enable the configuration of the electrical structures and / or the logic gates to form circuits for performing desired operations. The logic gate circuitry 1008 may include other electrical structures such as look-up tables (LUTs), registers (e.g., flip-flops or latches), multiplexers, etc.

[0080] The configurable interconnects 1010 of the illustrated example are conductive paths, traces, vias, or the like that may include electrically controllable switches (e.g., transistors) whose state may be changed by programming (e.g., using an HDL instruction language) to enable or disable one or more connections between one or more of the logic gate circuitry 1008 to program desired logic circuits.

[0081] The memory circuitry 1012 of the illustrated example is structured to store one or more results of one or more of the operations performed by the corresponding logic gates. The memory circuitry 1012 may be implemented by registers or the like. In the illustrated example, the memory circuitry 1012 is distributed among the logic gate circuitry 1008 to facilitate access and increase execution speed.

[0082] The exemplary FPGA circuit arrangement 1000 from Fig. 10 also includes example circuitry 1014 for dedicated operations. In this example, the circuitry 1014 for dedicated operations includes special-purpose circuitry 1016 that can be invoked to implement frequently used functions to avoid having to program those functions in the field. Examples of such special-purpose circuitry 1016 include memory control circuitry (e.g., DRAM), PCIe control circuitry, clock circuitry, transceiver circuitry, memory, and multiplier-accumulator circuitry. Other types of special-purpose circuitry may also be present. In some examples, the FPGA circuitry 1000 may also include example programmable general-purpose circuitry 1018, such as an example CPU 1020 and / or an example DSP 1022.Additionally or alternatively, there may also be further programmable general-purpose circuitry 1018, such as a GPU, an XPU, etc., which may be programmed to perform further operations.

[0083] Although Fig. 9 and Fig. 10 two exemplary implementations of the processor circuitry 812 from Fig. 8, many other approaches are conceivable. For example, a modern FPGA circuitry, as mentioned above, may include an on-board CPU such as one or more of the exemplary CPUs 1020 of Fig. 10. Thus, the processor circuit arrangement 812 may be Fig. 8 additionally by combining the exemplary microprocessor 900 from Fig. 9 and the exemplary FPGA circuit arrangement 1000 from Fig. 10. In some of these hybrid examples, a first part of the flowchart described in Fig. 7 by one or more of the cores 902 of Fig. 9, a second part of the flowchart from Fig. 7 can be executed by the FPGA circuitry 1000 of Fig. 10, and / or a third part of the flowchart shown in Fig. The machine-readable instructions shown in Figure 7 can be executed by an ASIC. It is understood that the entire circuit arrangement consists of Fig. 1 or a part thereof may be instantiated simultaneously or at different times. For example, the entire circuit arrangement or a part thereof may be instantiated in one or more threads that execute simultaneously and / or in series. Furthermore, in some examples, the entire circuit arrangement may consist of Fig. 1 or a part thereof may be implemented in one or more virtual machines and / or one or more containers running on the microprocessor.

[0084] In some examples, the processor circuitry 812 may be comprised of Fig. 8 in one or more packages. Thus, the microprocessor 900 can be made up of Fig. 9 and / or the FPGA circuit arrangement 1000 from Fig. 11, for example, in one or more packages. In some examples, an XPU may be implemented by the processor circuitry 812 of Fig. 8, which can be located in one or more packages. For example, the XPU can contain a CPU in one package, a DSP in another package, a GPU in yet another package, and an FPGA in yet another package.

[0085] A block diagram illustrating an example software distribution platform 1105 for distributing software, such as the example machine-readable instructions 832 of Fig. 8, on hardware devices owned and / or operated by third parties, is in Fig. 11. The example software distribution platform 1105 may be implemented by any computer server, data facility, cloud service, etc. capable of storing and transmitting software to other computing devices. The third parties may be customers of the entity that owns and / or operates the software distribution platform 1105. The entity that owns and / or operates the software distribution platform 1105 may be, for example, a developer, a vendor, and / or a licensor of software, such as the example machine-readable instructions 832 of Fig. 8. The third parties may be consumers, users, retailers, OEMs, etc., who acquire and / or license the software for use and / or resale and / or sublicensing. In the illustrated example, the software distribution platform 1105 includes one or more servers and one or more storage devices. The storage devices store the machine-readable instructions 832, which correspond to the example machine-readable instructions 700 of Fig. 7, as described above. The one or more servers of the example software distribution platform 1105 are in communication with an example network 1110, which may correspond to one or more of the Internet and / or example networks described above. In some examples, the one or more servers respond to requests to transfer the software to a requesting party as part of a commercial transaction. Payment for the delivery, sale, and / or license of the software may be processed through the one or more servers of the software distribution platform and / or through a third-party payment entity. The servers enable purchasers and / or licensors to download the machine-readable instructions 832 from the software distribution platform 1105. For example, the software corresponding to the example machine-readable instructions 700 of Fig. 7, may be downloaded to the example processor platform 800, which is to execute the machine-readable instructions 832 to implement the image processing system 100. In some examples, one or more servers of the software distribution platform 1105 periodically offer, transmit, and / or enforce updates to the software (e.g., the example machine-readable instructions 832 of Fig. 8) to ensure that improvements, patches, updates, etc. are distributed and applied to the software on the end-user devices.

[0086] From the foregoing, it can be seen that exemplary systems, methods, apparatus, and articles of manufacture have been disclosed that automatically process images based on the presence or absence of a face mask to generate images with the correct color and brightness of the facial skin of individuals wearing face masks. False color shifts and brightness levels produced by conventional image processing that does not account for the presence of a face mask are avoided. Furthermore, the examples disclosed herein do not require the large processing bandwidths and other resources consumed by techniques that rely solely on neural networks.In this way, the disclosed systems, methods, devices, and articles of manufacture improve the efficiency of using a computing device, and accordingly, the disclosed systems, methods, devices, and articles of manufacture are directed to one or more improvements in the operation of a machine, such as a computer or other electronic and / or mechanical device.

[0087] Example methods, apparatus, systems, and articles of manufacture for automatically processing an image based on the detection of a face mask are disclosed. Example 1 includes an apparatus comprising: at least one memory; machine-readable instructions; and processor circuitry for instantiating or executing the machine-readable instructions to: determine a feature of an upper portion of a person's face in an image; determine a feature of a lower portion of the person's face; calculate a distance between the upper portion feature and the lower portion feature; compare the distance to a threshold; and identify the presence of a face mask if the distance is greater than the threshold.

[0088] Example 2 includes the device of Example 1, wherein the feature is at least one of a hue, a color saturation, and a brightness value.

[0089] Example 3 includes the apparatus of example 1 and / or 2, wherein the processor circuitry is to: convert the RGB color space of the image to an HSV color space and determine the upper region feature and the lower region feature based on the HSV color space.

[0090] Example 4 includes the device of any one of Examples 1-3, wherein the upper range feature is at least an average of a hue over the upper range, an average of a chroma over the upper range, and / or an average of a brightness over the upper range, and wherein the lower range feature is at least an average of a hue over the lower range, an average of a chroma over the lower range, and / or an average of a brightness over the lower range.

[0091] Example 5 includes the device of any one of Examples 1-4, wherein the upper region feature is based on an average of hue over the upper region, an average of chroma over the upper region, and an average of brightness over the upper region, and wherein the lower region feature is based on an average of hue over the lower region, an average of chroma over the lower region, and an average of brightness over the lower region.

[0092] Example 6 includes the apparatus of any one of Examples 1-5, wherein the processor circuitry is to process the upper region with automatic exposure and skip the lower region when the face mask is identified.

[0093] Example 7 includes the apparatus of any one of Examples 1-6, wherein the processor circuitry is to process the upper region with automatic white balance and skip the lower region when the face mask is identified.

[0094] Example 8 includes the apparatus of any one of Examples 1-7, wherein the processor circuitry is to: identify a boundary line on the person's face and, based on the boundary line, divide the face into the upper region and the lower region.

[0095] Example 9 includes an apparatus comprising: at least one memory;

[0096] machine-readable instructions and processor circuitry for instantiating or executing the machine-readable instructions to: determine a feature of an upper region of a person's face in an image; determine a feature of a lower region of the person's face; map the upper region feature and the lower region feature to a color diagram including a facial skin tone cone; and identify the presence of a face mask when the upper region feature lies within the facial skin tone cone and the lower region feature does not lie within the skin tone cone.

[0097] Example 10 includes the device of Example 9, wherein the feature is at least one of a hue, a color saturation, and a brightness value.

[0098] Example 11 includes the apparatus of example 9 and / or 10, wherein the processor circuitry is to: convert the RGB color space of the image to an HSV color space and determine the upper region feature and the lower region feature based on the HSV color space.

[0099] Example 12 includes the device of any one of Examples 9-11, wherein the upper range feature is based on an average of a hue over the upper range, an average of a chroma over the upper range, and an average of a brightness over the upper range, and wherein the lower range feature is based on an average of a hue over the lower range, an average of a chroma over the lower range, and an average of a value over the lower range.

[0100] Example 13 includes the apparatus of any one of Examples 9-12, wherein the processor circuitry is to avoid the lower region and process the upper region with automatic exposure when the face mask is identified.

[0101] Example 14 includes the apparatus of any one of Examples 9-13, wherein the processor circuitry is to avoid the lower region and process the upper region with automatic white balance when identifying the face mask.

[0102] Example 15 includes the apparatus of any of Examples 9-14, wherein the processor circuitry is to: determine a distance between the upper region feature and the lower region feature; and verify the presence of the face mask based on the distance.

[0103] Example 16 includes a non-transitory machine-readable storage medium comprising instructions that, when executed, cause programmable circuitry to at least: map a feature of an upper portion of a person's face in an image to a color map including a facial skin tone cone; map a feature of a lower portion of the face to the color map; and identify the presence or absence of a face mask based on the respective positions of the upper portion feature and the lower portion feature with respect to the facial skin tone cone.

[0104] Example 17 includes the storage medium of Example 16, wherein the instructions cause the programmable circuitry to determine a position of a boundary between the upper region and the lower region.

[0105] Example 18 includes the storage medium of example 16 and / or 17, wherein the instructions cause the programmable circuitry to: convert the RGB color space of pixels in the image to an HSV color space; and determine the upper region feature and the lower region feature based on the HSV color space.

[0106] Example 19 includes the storage medium of any one of Examples 16-18, wherein the instructions cause the programmable circuitry to: determine the upper region characteristic by aggregating a hue average, a chroma average, and a brightness average of a plurality of pixels in the upper region, and determine the lower region characteristic by aggregating a hue average, a chroma average, and a brightness average of a plurality of pixels in the lower region.

[0107] Example 20 includes the storage medium of any of Examples 16-19, wherein the instructions cause the programmable circuitry to identify the presence of the face mask when the upper region feature is within the facial skin tone cone and the lower region feature is not within the skin tone cone.

[0108] Example 21 includes the storage medium of any of Examples 16-20, wherein the instructions cause the programmable circuitry to identify the absence of the face mask when the upper region feature is not within the facial skin tone cone.

[0109] Example 22 includes the storage medium of any of Examples 16-21, wherein the instructions cause the programmable circuitry to: calculate a distance between the upper region feature and the lower region feature and verify the presence or absence of the face mask based on the distance.

[0110] Example 23 includes the storage medium of any of Examples 16-22, wherein the instructions cause the programmable circuitry to: compare the distance to a distance threshold and identify the presence of the face mask if the distance is greater than the threshold.

[0111] Example 24 includes a method for processing images based on the detection of a face mask, the method comprising: identifying, by executing instructions with a processor, an upper region of a face and a lower region of a face; calculating, by executing instructions with the processor: a hue mean (Hu), a chroma mean (Su), and a luminance mean (Vu) for the upper region; a hue mean (HI), a chroma mean (SI), and a luminance mean (VI) for the lower region; an HSV combination for the upper region: Pu (Hu, Su, Vu) and an HSV combination for the lower region: PI (HI, SI, VI); mapping, by executing instructions with the processor, Pu and PI to a graph including a facial skin tone cone; and identifying, by executing instructions with the processor, the presence or absence ofAbsence of the facial mask based on a respective position of Pu on the image and of PI on the image with respect to the facial skin tone cone.

[0112] Example 25 includes the method of Example 24 and further comprises: by executing instructions with the processor, identifying the presence of the face mask when Pu is within the facial skin tone cone and PI is not within the skin tone cone; or by executing instructions with the processor, identifying the presence of the face mask when (a) Pu is within the facial skin tone cone, (b) Pl is within the facial skin tone cone, and (c) a calculated distance between one or more of the values Hu, Su, Vu and one or more of the corresponding values HI, SI, VI is greater than a distance threshold.

[0113] The following claims are hereby incorporated by reference into this Detailed Description. While certain exemplary systems, methods, devices, and articles of manufacture have been disclosed herein, the scope of this patent is not so limited. Rather, this patent covers all systems, methods, devices, and articles of manufacture that reasonably fall within the scope of the claims of this patent.

Claims

[1] Device comprising: at least one memory; machine-readable instructions and processor circuitry for instantiating and / or executing the machine-readable instructions to: to determine a feature of an upper region of a person's face in an image; to determine a feature of a lower part of the person's face; calculate a distance between the upper region feature and the lower region feature; to compare the distance with a threshold value and to identify the presence of a face mask when the distance is greater than the threshold. [2] Device according to claim 1, wherein the feature is at least one of a hue, a color saturation and a brightness value. [3] The apparatus of claim 2, wherein the processor circuitry is arranged to: convert the RGB color space of the image into an HSV color space and to determine the upper range feature and the lower range feature based on the HSV color space. [4] The device of claim 1, wherein the upper range feature is at least an average of a hue over the upper range, an average of a chroma over the upper range, and / or an average of a brightness over the upper range, and wherein the lower range feature is at least an average of a hue over the lower range, an average of a chroma over the lower range, and / or an average of a brightness over the lower range. [5] The device of claim 1, wherein the upper range feature is based on an average of hue over the upper range, an average of chroma over the upper range, and an average of brightness over the upper range, and wherein the lower range feature is based on an average of hue over the lower range, an average of chroma over the lower range, and an average of brightness over the lower range. [6] The apparatus of claim 1, wherein the processor circuitry is to process the upper region with automatic exposure and skip the lower region when the face mask is identified. [7] The apparatus of claim 1, wherein the processor circuitry is to process the upper region with automatic white balance and skip the lower region when the face mask is identified. [8] The apparatus of claim 1, wherein the processor circuitry is arranged to: to identify a boundary line on the person’s face and to divide the face into the upper and lower areas based on the boundary line. [9] Device comprising: at least one memory; machine-readable instructions and processor circuitry for instantiating and / or executing the machine-readable instructions to: to determine a feature of an upper region of a person's face in an image; to determine a feature of a lower part of the person's face; map the upper region feature and the lower region feature onto a color diagram containing a facial skin tone cone, and to identify the presence of a facial mask when the upper region feature lies within the facial skin tone cone and the lower region feature does not lie within the skin tone cone. [10] Device according to claim 9, wherein the feature is at least one of a hue, a color saturation and a brightness value. [11] Apparatus according to claim 9, wherein the processor circuitry is arranged to: convert the RGB color space of the image into an HSV color space and to determine the upper range feature and the lower range feature based on the HSV color space. [12] The apparatus of claim 9, wherein the upper range feature is based on an average of hue over the upper range, an average of chroma over the upper range, and an average of brightness over the upper range, and wherein the lower range feature is based on an average of hue over the lower range, an average of chroma over the lower range, and an average of brightness over the lower range. [13] The apparatus of claim 9, wherein the processor circuitry is to avoid the lower region and process the upper region with automatic exposure when the face mask is identified. [14] The apparatus of claim 9, wherein the processor circuitry is to avoid the lower region and process the upper region with automatic white balance when identifying the face mask. [15] Apparatus according to claim 9, wherein the processor circuitry is arranged to: to determine a distance between the feature of the upper area and the feature of the lower area and to verify the presence of the face mask based on the distance. [16] Non-transitory machine-readable storage medium comprising instructions which, when executed, cause a programmable circuit arrangement to at least: map a feature of an upper region of a person's face in an image to a color diagram including a facial skin tone cone; to map a feature of a lower area of the face on the color diagram and to identify the presence or absence of a face mask based on the respective positions of the upper region feature and the lower region feature with respect to the facial skin tone cone. [17] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to determine a position of a boundary between the upper region and the lower region. [18] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to: convert the RGB color space of pixels in the image into an HSV color space and to determine the upper range feature and the lower range feature based on the HSV color space. [19] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to: to determine the upper region feature by aggregating a hue average, a chroma average and a brightness average of several pixels in the upper region and to determine the lower region feature by aggregating a hue mean, a chroma mean, and a brightness mean of several pixels in the lower region. [20] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to identify the presence of the face mask when the upper region feature is within the facial skin tone cone and the lower region feature is not within the skin tone cone. [21] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to identify the absence of the face mask when the upper region feature is not within the facial skin cone. [22] The storage medium of claim 16, wherein the instructions cause the programmable circuitry to: to calculate a distance between the feature of the upper area and the feature of the lower area and to verify the presence or absence of the face mask based on the distance. [23] The storage medium of claim 22, wherein the instructions cause the programmable circuitry to: to compare the distance with a distance threshold and to identify the presence of the face mask when the distance is greater than the threshold. [24] A method for processing images based on the detection of a face mask, the method comprising: Identifying a top portion of a face and a bottom portion of a face by executing instructions with a processor; by executing instructions with the processor Calculate: a hue average (Hu), a chroma average (Su) and a brightness average (Vu) for the upper range; a hue mean (Hl), a chroma mean (SI) and a brightness mean (VI) for the lower range; an HSV combination for the upper range: Pu (Hu, Su, Vu) and a HSV combination for the lower range: PI (HI, SI, VI); by executing instructions with the processor mapping Pu and Pl to a diagram including a facial skin tone cone; and by executing instructions with the processor, identifying the presence or absence of the face mask based on a respective position of Pu on the image and of PI on the image with respect to the facial skin tone cone. [25] The method of claim 24, further comprising: (1) by executing instructions with the processor, identifying the presence of the face mask when Pu is within the facial skin tone cone and PI is not within the skin tone cone; or (2) by executing instructions with the processor, identifying the presence of the face mask when (a) Pu is within the facial skin tone cone, (b) Pl is within the facial skin tone cone, and (c) a calculated distance between one or more of the values Hu, Su, Vu and one or more of the corresponding values HI, SI, VI is greater than a distance threshold.

Images