A face recognition processing method and device
By performing Gaussian noise reduction and HSV color model detection on facial images in the cafeteria settlement system, and combining the Adaboost algorithm to determine facial integrity, the problem of facial recognition failure was solved, and the recognition success rate and user experience were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA CONSTRUCTION BANK
- Filing Date
- 2023-09-15
- Publication Date
- 2026-06-05
Smart Images
Figure CN117079334B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of biometric recognition technology in the field of artificial intelligence, and in particular to a face recognition processing method and apparatus. Background Technology
[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. With its development, facial recognition technology has become widely used across various industries due to its advantages such as data accuracy, high security, and ease of use, including unmanned vending machines, time and attendance machines, access control systems, and payment processing equipment.
[0003] Taking the cafeteria payment scenario as an example, the cafeteria payment app integrates a face algorithm SDK (Software Development Kit). Before payment, a liveness detection is performed. After passing the detection, the captured face image is sent to the face database for a 1:N search to obtain user information before payment settlement.
[0004] However, during peak dining hours in the cafeteria, with a large flow of diners, the facial images captured after the SDK's liveness detection may have the following issues: 1. Facial movement may cause the captured facial outline to be blurry with no clear facial contours; 2. The actual size of the face in the image may be too small, failing to meet the requirements for financial payments; 3. Partial occlusion of the face may cause the facial image to fail the face database verification, resulting in facial recognition failure, requiring a second liveness detection and capture of the face image. Before the second liveness detection, users need to wait for the network request response time and for the device screen to display a countdown indicating recognition failure, reducing the user experience of paying for meals with facial recognition. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a face recognition processing method and apparatus, which can at least solve the problems in the prior art where face movement causes blurred face outlines, excessively small face size, or occlusion, resulting in face recognition failure and affecting users' face payment.
[0006] To achieve the above objectives, according to one aspect of the present invention, a face recognition processing method is provided, comprising:
[0007] Receive the original face image captured by the face capture interface, input the original face image into a preset color model, and obtain the hue value and brightness value of each pixel in the original face image;
[0008] A set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range is determined to obtain the skin color region in the original face image, and the skin color region is then processed for review.
[0009] In response to the skin color region passing the review, a preset face detection model is invoked to identify whether the face in the original face image is complete. In response to the recognition result being complete, the original face image is transmitted to the face recognition interface for processing.
[0010] Optionally, before inputting the original face image into a preset color model, the method further includes:
[0011] The original face image is subjected to Gaussian noise reduction to obtain a denoised face image.
[0012] Optionally, before performing Gaussian noise reduction on the original face image, the method further includes:
[0013] Using the top left corner of the original face image as the origin, a rectangle is used to select the face region in the original face image to obtain the region of interest.
[0014] Optionally, performing Gaussian noise reduction on the original face image to obtain a denoised face image includes:
[0015] For each pixel in the original face image, determine the original pixel value and weight, and determine the pixel value of each other pixel in the neighborhood and the distance to each pixel;
[0016] Based on the preset correspondence between distance and weight, the weight corresponding to each distance is determined, and then the pixel values of each pixel and all other pixels in the neighborhood are weighted and summed to obtain the weighted summation result.
[0017] The weights of each pixel and all other pixels in its neighborhood are summed. The quotient of the weighted summation result and the sum of the weights is calculated. The quotient is then rounded down to obtain the noise-reduced pixel value of each pixel.
[0018] Optionally, after obtaining the skin color region in the original face image, the method further includes:
[0019] A binary face image is generated based on the skin color region, and the pixels in the skin color region are assigned a first preset pixel value, and the pixels in other regions are assigned a second preset pixel value.
[0020] The binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, and the largest connected region is taken as the connected skin color region.
[0021] Optionally, the review process for the skin-colored area includes:
[0022] Calculate the number of first pixels in the skin color region. If the number of first pixels is less than a preset threshold for the number of pixels in the skin color region, determine that the original face image has not passed the review, and issue a command to re-acquire the face image to the face acquisition interface.
[0023] Optionally, the review process for the skin-colored area includes:
[0024] The single-pixel edge of the skin-colored region is drawn using an edge detection algorithm, and the edge pixels are assigned a third preset pixel value, while the other pixels are assigned a second preset pixel value.
[0025] Calculate the number of second pixels at the edge of the single pixel. If the number of second pixels is less than a preset threshold for the number of edge pixels, determine that the original face image has not passed the review and issue a command to re-acquire the face image to the face acquisition interface.
[0026] To achieve the above objectives, according to another aspect of the present invention, a face recognition processing apparatus is provided, comprising:
[0027] The face image preprocessing module is used to receive the original face image acquired by the face acquisition interface, input the original face image into a preset color model, and obtain the hue value and brightness value of each pixel in the original face image.
[0028] The skin color detection module is used to determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the original face image and to perform review processing on the skin color region.
[0029] The face detection module is used to respond to the skin color region passing the review, call the preset face detection model to identify whether the face in the original face image is complete, and in response to the recognition result being complete, transmit the original face image to the face recognition interface for processing.
[0030] Optionally, the face image preprocessing module is further used for:
[0031] The original face image is subjected to Gaussian noise reduction to obtain a denoised face image.
[0032] Optionally, the face image preprocessing module is further used for:
[0033] Using the top left corner of the original face image as the origin, a rectangle is used to select the face region in the original face image to obtain the region of interest.
[0034] Optionally, the face image preprocessing module is used for:
[0035] For each pixel in the original face image, determine the original pixel value and weight, and determine the pixel value of each other pixel in the neighborhood and the distance to each pixel;
[0036] Based on the preset correspondence between distance and weight, the weight corresponding to each distance is determined, and then the pixel values of each pixel and all other pixels in the neighborhood are weighted and summed to obtain the weighted summation result.
[0037] The weights of each pixel and all other pixels in its neighborhood are summed. The quotient of the weighted summation result and the sum of the weights is calculated. The quotient is then rounded down to obtain the noise-reduced pixel value of each pixel.
[0038] Optionally, the skin color detection module is further used for:
[0039] A binary face image is generated based on the skin color region, and the pixels in the skin color region are assigned a first preset pixel value, and the pixels in other regions are assigned a second preset pixel value.
[0040] The binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, and the largest connected region is taken as the connected skin color region.
[0041] Optionally, the skin color detection module is further used for:
[0042] Calculate the number of first pixels in the skin color region. If the number of first pixels is less than a preset threshold for the number of pixels in the skin color region, determine that the original face image has not passed the review, and issue a command to re-acquire the face image to the face acquisition interface.
[0043] Optionally, an edge detection module is also included for:
[0044] The single-pixel edge of the skin-colored region is drawn using an edge detection algorithm, and the edge pixels are assigned a third preset pixel value, while the other pixels are assigned a second preset pixel value.
[0045] Calculate the number of second pixels at the edge of the single pixel. If the number of second pixels is less than a preset threshold for the number of edge pixels, determine that the original face image has not passed the review and issue a command to re-acquire the face image to the face acquisition interface.
[0046] To achieve the above objectives, according to another aspect of the present invention, a face recognition processing electronic device is provided.
[0047] The electronic device of this invention includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the face recognition processing methods described above.
[0048] To achieve the above objectives, according to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the face recognition processing methods described above.
[0049] To achieve the above objectives, according to another aspect of the present invention, a computing program product is provided. One such computing program product includes a computer program that, when executed by a processor, implements the face recognition processing method provided in the present invention.
[0050] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: For face images with low resolution and few pixels output by the SDK, a face image evaluation step is set up: the face image is denoised, skin color regions are identified, the number of face pixels is calculated to see if it meets the threshold, the number of pixels at the edge of a single face pixel meets the threshold, and the face is judged to be complete. If the evaluation passes, the face image is sent to the face database for recognition; if the evaluation fails, the face image is re-collected, thereby improving the success rate of face database recognition and improving the user's face payment experience.
[0051] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description
[0052] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:
[0053] Figure 1 This is a schematic diagram of the main process of a face recognition processing method according to an embodiment of the present invention;
[0054] Figure 2 This is a flowchart illustrating an optional face recognition processing method according to an embodiment of the present invention;
[0055] Figure 3 This is a schematic diagram illustrating the pixel values used for noise reduction calculation;
[0056] Figure 4 This is a flowchart illustrating another optional face recognition processing method according to an embodiment of the present invention;
[0057] Figure 5 This is a flowchart illustrating a specific face recognition processing method according to an embodiment of the present invention;
[0058] Figure 6 This is a schematic diagram of the main modules of a face recognition processing device according to an embodiment of the present invention;
[0059] Figure 7 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;
[0060] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present invention, such as a mobile device or server. Detailed Implementation
[0061] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0062] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The collection, analysis, use, transmission, and storage of user personal information involved in the technical solutions of the present invention all comply with relevant laws and regulations, are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to such personal information data, ensure that personnel authorized to access personal information data comply with relevant laws and regulations, and ensure the security of user personal information.
[0063] Once this user's personal information data is no longer needed, the risk should be minimized by restricting or even prohibiting data collection and / or deleting the data. Where applicable, including in certain relevant applications, user privacy should be protected by de-identifying the data, such as by removing specific identifiers (e.g., date of birth), controlling the amount or specificity of the stored data (e.g., collecting location data at the city level rather than the specific address level), controlling how the data is stored, and / or other de-identification methods.
[0064] See Figure 1 The diagram shows the main flowchart of a face recognition processing method provided by an embodiment of the present invention, which includes the following steps:
[0065] S101: Receive the original face image collected by the face acquisition interface, input the original face image into a preset color model, and obtain the hue value and brightness value of each pixel in the original face image;
[0066] S102: Determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the original face image, and perform review processing on the skin color region;
[0067] S103: In response to the skin color region passing the review, a preset face detection model is invoked to identify whether the face in the original face image is complete. In response to the recognition result being complete, the original face image is transmitted to the face recognition interface for processing.
[0068] In the above implementation, this solution takes the canteen settlement scenario as an example. The canteen settlement system addresses the difficulties and pain points of traditional canteens, such as inconvenient recharge and settlement, by providing intelligent and convenient functions such as cardless dining, mobile phone ordering, time-sharing management, and meal subsidy settings, truly realizing the empowerment of canteen construction and management by technology.
[0069] In step S101, the SDK (referred to as the Face Acquisition Interface in this solution) is mainly used for liveness detection and acquiring raw face images, outputting the raw face images that have passed liveness detection to the application. Before sending the face image to the face database for a 1:N search, this solution develops a rapid face image evaluation step between the SDK and the face database (referred to as the Face Recognition Interface in this solution). This step is usually handled by the mobile app. After rapid evaluation, the face image is then sent to the face database via the network for a 1:N search, thereby increasing the face database recognition rate and improving the user's face payment experience.
[0070] The system receives the original face image from the SDK, which is approximately 440*440 pixels in size. Considering that each color is represented by hue (H), saturation (S), and value (V), this solution preferably uses the HSV color model, which uses three channels to represent color information: hue (H), saturation (S), and value (V). However, in practice, saturation (S) has limited use in face recognition; therefore, this solution only considers the H and V channels. Using the HSV color model, the H and V values of each pixel in the original face image can be obtained.
[0071] For step S102, this scheme pre-sets the threshold ranges of the H and V channels in the HSV color model, such as H channel [3,14] and V channel [175,256], to perform skin color detection processing in order to detect the skin color area in the original face image.
[0072] After obtaining the H and V component values of each pixel in the original face image in the HSV color model through the above steps, the skin color regions are checked using 3 <= H <= 14 and 175 <= V <= 256. Specifically, the set of pixels with H values in the range [3, 14] and V values in the range [175, 256] is determined to obtain the skin color regions. In practice, due to the influence of clothing and background, such as wearing glasses or local reflections on the face, the original face image will generate many disconnected skin color regions, with the face being the largest skin color region.
[0073] After detecting the skin-colored region, a binary face image is generated. The pixel value in the skin-colored region is set to 255 (i.e., the first preset pixel value, corresponding to white), and the remaining pixels in other regions are set to 0 (i.e., the second preset pixel value, corresponding to black). Then, the binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, such as eliminating some scattered small skin-colored regions, removing the disconnection caused by local reflections, and connecting the upper and lower parts of the face by removing the gaps caused by glasses. The largest connected region is calculated as the face, and the pixel values of other regions are all assigned to 0.
[0074] Given the limited number of pixels and low resolution of facial images captured by mobile payment devices, facial image detection is necessary. There are various facial image detection algorithms, including neural network models and classic machine learning algorithms. Considering the initial goal of quickly evaluating the quality of facial images captured by the SDK, this solution preferentially adopts classic machine vision algorithms to quickly evaluate the quality of facial images.
[0075] In step S103, if the skin color region passes the detection, the original face image is transmitted to the Adaboost algorithm module. The Adaboost algorithm module receives the original face image, and the cafeteria settlement app loads the trained Adaboost face detection model to determine if the face in the original face image is complete. If the face is incomplete, the program logic sends a command to the SDK to re-acquire the face image, returning to the SDK's liveness detection stage. Otherwise, it is considered to have passed face detection. Then, the application encodes the original face image transmitted by the SDK in a fixed format (e.g., base64) and sends it to the face database over the network.
[0076] The method provided in the above embodiments sets up a face image evaluation step for face images with low resolution and few pixels output by the SDK: identifying skin color areas and detecting and judging whether the face is complete. If the evaluation passes, the face image is sent to the face database for recognition; if the evaluation fails, the face image is re-collected, thereby improving the success rate of face database recognition and improving the user's face payment experience.
[0077] See Figure 2The diagram illustrates an optional face recognition processing method according to an embodiment of the present invention, including the following steps:
[0078] S201: Receive the original face image collected by the face acquisition interface, perform Gaussian noise reduction processing on the original face image, and obtain a noise-reduced face image;
[0079] S202: Input the denoised face image into a preset color model to obtain the hue value and brightness value of each pixel in the denoised face image;
[0080] S203: Determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the denoised face image, and perform review processing on the skin color region;
[0081] S204: In response to the skin color region passing the review, a preset face detection model is invoked to identify whether the face in the denoised face image is complete. In response to the recognition result being complete, the original face image is transmitted to the face recognition interface for processing.
[0082] In the above embodiments, for steps S201 to S203, the original face image transmitted by the SDK is received. The original face image output by the SDK is approximately 440*440 pixels. Gaussian noise reduction processing is performed on the original face image to filter out abnormal noise points that affect face detection.
[0083] Image denoising involves using filtering algorithms to remove abnormal pixel values from an image. This solution preferably uses a Gaussian filtering algorithm, which eliminates the need to identify abnormal pixels and smooths the overall image, effectively removing salt-and-pepper noise (also known as impulse noise). Specifically, each pixel in the image is reassigned a new value. The new pixel value is calculated by weighting the original pixel value and the pixel values of all its neighboring pixels, thus smoothing the pixel values of the entire image. The neighborhood can be a 3x3 pixel or a 5x5 pixel neighborhood; there are no restrictions. The weights depend on the distance from each neighboring pixel to the center pixel; closer pixels have higher weights, and farther pixels have lower weights.
[0084] See Figure 3 As shown, the original pixel value of the center pixel is 90, with a weight of 1. Its neighborhood is a 3x3 pixel area. The pixels in the top, bottom, left, and right directions have a value of 91, with a weight of 0.9. The pixels in the top left, bottom left, top right, and bottom right directions have a value of 92, with a weight of 0.5. It can be seen that the closer to the center pixel, the larger the weight. The weighted average of these pixel values, after Gaussian filtering, yields the new pixel value of the center pixel.
[0085] (91*1+01*0.9*4+92*0.5*4) / (1+0.9*4+0.5*4)≈91
[0086] It should be noted that the correspondence between the preset distance and weight in this scheme can be pre-configured by the staff. For example, in a neighborhood composed of 5x5 pixels, the weights of the four directions of the upper left, lower left, upper right, and lower right can be 0.1, 0.2, etc., and can be configured arbitrarily, but the weights of symmetrical positions must be consistent.
[0087] While existing image filtering methods include mean filtering, these methods are fast but prone to losing feature points and edge information in the image. Therefore, this solution preferably uses the Gaussian filtering algorithm.
[0088] In step S204, the Adaboost algorithm module receives the denoised face image. The cafeteria settlement app loads the trained Adaboost face detection model to determine whether the face in the denoised face image is complete. If the face is incomplete, the program logic returns to the SDK liveness detection stage. Otherwise, it is considered to have passed the face detection. Then, the application encodes the original face image transmitted by the SDK in a fixed format and sends it to the face database via the network for face recognition.
[0089] The method provided in the above embodiments performs Gaussian noise reduction on the original face image to filter out abnormal noise points that affect face detection, thereby reducing the computational load of subsequent processing.
[0090] See Figure 4 The diagram illustrates another optional face recognition processing method according to an embodiment of the present invention, including the following steps:
[0091] S401: Receive the original face image acquired by the face acquisition interface, and use a rectangle to select the face region in the original face image with the upper left corner of the original face image as the origin to obtain the region of interest;
[0092] S402: Perform Gaussian noise reduction on the region of interest to obtain a denoised face image. Input the denoised face image into a preset color model to obtain the hue value and brightness value of each pixel in the denoised face image.
[0093] S403: Determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the denoised face image, and perform review processing on the skin color region;
[0094] S404: In response to the skin color region passing the review, a preset face detection model is invoked to identify whether the face in the denoised face image is complete. In response to the recognition result being complete, the original face image is transmitted to the face recognition interface for processing.
[0095] In the above embodiments, for steps S401 to S403, to further reduce the computational load, this scheme can also use the upper left corner of the original face image as the origin, with the horizontal direction to the left as the positive x-axis and the vertical direction downwards as the positive y-axis. A rectangular selection method is used to select the face region in the original face image, with the face generally located within the range (100, 20, 350, 350). This region is set as the ROI (region of interest) of the original face image. This effectively avoids interference from clothing and backgrounds with similar skin tones, while significantly reducing the computational load and accelerating image processing. Subsequently, Gaussian noise reduction is performed only on the ROI region to filter out abnormal noise points in that region.
[0096] In machine vision and image processing, the region to be processed in an image is delineated using rectangles, circles, ellipses, irregular polygons, etc., and is called the Region of Interest (ROI). In the field of image processing, the ROI is an image region selected from the original face image. This region is the focus of image analysis. Delineating this region for further processing can reduce processing time and increase accuracy.
[0097] This solution pre-sets the threshold ranges for the H and V components in the HSV color model, such as H channel [3,14] and V channel [175,256], to perform skin color detection processing and detect skin-colored areas in the ROI region.
[0098] After obtaining the denoised ROI region through the above steps, the H and V component values of each pixel within the denoised ROI region in the HSV color model are obtained. Skin color regions are then examined using 3 <= H <= 14 and 175 <= V <= 256. In practice, due to the influence of clothing and background, many disconnected skin color regions are generated within the ROI image, with the face being the largest.
[0099] For step S404, if the skin color area passes the review and detection, the aforementioned output denoised ROI area is sent to the Adaboost algorithm module, which is used to detect whether the face in the denoised ROI area is complete.
[0100] The method provided in the above embodiments extracts the ROI region from the original face image, and then processes the entire process using the ROI region as the object, which further reduces the amount of computation.
[0101] See Figure 5 The diagram illustrates a specific face recognition processing method according to an embodiment of the present invention, including the following steps:
[0102] S501: Receive the original face image acquired by the face acquisition interface, and use a rectangle to select the face region in the original face image with the upper left corner of the original face image as the origin to obtain the region of interest;
[0103] S502: Perform Gaussian noise reduction on the region of interest to obtain a denoised face image. Input the denoised face image into a preset color model to obtain the hue value and brightness value of each pixel in the denoised face image.
[0104] S503: Determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the denoised face image, and calculate the number of first pixels in the skin color region;
[0105] S504: In response to the fact that the number of the first pixel is less than the preset threshold for the number of pixels in the skin color region, it is determined that the original face image has failed the evaluation, and a command to re-acquire the face image is sent to the face acquisition interface.
[0106] S505: In response to the first pixel count being greater than or equal to a preset threshold for the number of pixels in the skin color region, draw a single-pixel edge of the skin color region using an edge detection algorithm, assign a third preset pixel value to the edge pixels and a second preset pixel value to the other pixels, and calculate the second pixel count of the single-pixel edge.
[0107] S506: In response to the fact that the number of the second pixel is less than the preset threshold for the number of edge pixels, it is determined that the original face image has not passed the review, and a command to re-acquire the face image is sent to the face acquisition interface.
[0108] S507: In response to the second pixel count being greater than or equal to a preset edge pixel count threshold, the skin color region is determined to have passed the review. A preset face detection model is invoked to identify whether the face in the denoised face image is complete. In response to the recognition result being complete, the original face image is transmitted to the face recognition interface for processing.
[0109] In the above embodiments, steps S501 to S503 can be found in [reference needed]. Figure 4 The description shown is not repeated here.
[0110] For steps S504 to S507, due to the limited computing power of the mobile device's processor, it is impossible to use high-precision machine learning algorithms with high computational complexity and large computational volume to detect faces in each preview frame in real time. Therefore, in the cafeteria settlement scenario, the mobile app's face algorithm inevitably reduces detection accuracy while taking into account processing speed. The SDK may output face images with blurred face contours, small real face size, and occluded faces.
[0111] This solution includes a threshold comparison module to compare the number of pixels in the skin-tone region and the number of pixels at the edge of a single pixel in the skin-tone region of an actual face image. After processing to obtain the skin-tone region of the face, the skin-tone detection module calculates the number of the first pixel in the skin-tone region and transmits this number to the threshold comparison module. If the number of the first pixel is less than the preset threshold of 27000 for the number of pixels in the skin-tone region (this value is only an example and can be adjusted in practice), the original face image is considered to have failed the evaluation and is returned to the SDK liveness detection stage. Otherwise, it is considered to have passed the skin-tone detection stage, and the skin-tone region (or the binary face image after processing the skin-tone region) is transmitted to the edge detection module.
[0112] Edge detection methods can reflect the edge information of objects in an image. The edge detection module uses the Canny edge detection algorithm to draw single-pixel edges of the face image for skin-colored regions (or binary face images after skin-colored region processing). Edge pixels are assigned a value of 255 (i.e., the third preset pixel value), while all other pixel values are set to 0. The module then calculates the number of second-order pixels for each edge pixel and sends this number to the threshold comparison module. If the number of second-order pixels is less than the preset edge pixel count threshold of 1300 (this value is only an example and can be adjusted in practice), the original face image is considered to have failed the evaluation, and the program logic returns to the SDK liveness detection stage. Otherwise, it is considered to have passed the edge detection stage.
[0113] The above steps can largely identify whether a face meets the requirements, such as whether the face is wearing a mask, has a large proportion of the face, the face acquisition is unstable, the face acquisition is incomplete, or the face segmentation is not connected. Therefore, there is no need to consider whether the number of pixels in the length and width of the face meets the threshold.
[0114] The method provided in the above embodiments adds a skin color detection module and an edge detection module to preprocess the image. If the face image fails to pass the evaluation of these two modules, the evaluation process ends directly and there is no need to send it to the Ababoot algorithm module for processing, which can save more time and improve efficiency.
[0115] Current mobile processors have limited computing power, and their GPUs are insufficient to support complex neural networks for real-time face image detection. Therefore, to ensure real-time detection of all camera preview frames, machine learning algorithms with low time complexity and relatively low computational cost must be used, resulting in a loss of precision and accuracy. This means that the face images output by the SDK may not meet the requirements of the face database. Encoding and transmitting non-compliant face images to the face database reduces the success rate of face recognition, prolongs the time required for users to complete face-scanning payments, and degrades the user experience for dining with face-scanning payments.
[0116] To address the issue that the face recognition algorithm used in the SDK inevitably reduces recognition accuracy while balancing processing speed, the method provided in this invention first sets the Region of Interest (ROI) based on the characteristics of the face image output by the SDK. Gaussian noise reduction is then applied to the ROI to filter out abnormal pixel values caused by hardware or transmission processes. Next, edge detection and HSV color model skin color detection are performed on the ROI to exclude blurry faces or faces without clear facial contours. Finally, the Adaboost algorithm is used to detect whether the face in the denoised ROI is complete. This enables rapid evaluation of whether the face image output by the SDK meets the requirements of the face database, thereby increasing the face recognition pass rate, shortening the time for individual users to complete face payment, and improving the user experience for dining in cafeterias.
[0117] See Figure 6 The diagram shows a schematic representation of the main modules of a face recognition processing device 600 provided in an embodiment of the present invention, including:
[0118] The face image preprocessing module 601 is used to receive the original face image acquired by the face acquisition interface, input the original face image into a preset color model, and obtain the hue value and brightness value of each pixel in the original face image.
[0119] The skin color detection module 602 is used to determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the original face image and to perform review processing on the skin color region.
[0120] The face detection module 603 is used to, in response to the skin color region passing the review, call a preset face detection model to identify whether the face in the original face image is complete, and in response to the recognition result being complete, transmit the original face image to the face recognition interface for processing.
[0121] In the apparatus of this invention, the face image preprocessing module 601 is further used for:
[0122] The original face image is subjected to Gaussian noise reduction to obtain a denoised face image.
[0123] In the apparatus of this invention, the face image preprocessing module 601 is further used for:
[0124] Using the top left corner of the original face image as the origin, a rectangle is used to select the face region in the original face image to obtain the region of interest.
[0125] In the apparatus of this invention, the face image preprocessing module 601 is used for:
[0126] For each pixel in the original face image, determine the original pixel value and weight, and determine the pixel value of each other pixel in the neighborhood and the distance to each pixel;
[0127] Based on the preset correspondence between distance and weight, the weight corresponding to each distance is determined, and then the pixel values of each pixel and all other pixels in the neighborhood are weighted and summed to obtain the weighted summation result.
[0128] The weights of each pixel and all other pixels in its neighborhood are summed. The quotient of the weighted summation result and the sum of the weights is calculated. The quotient is then rounded down to obtain the noise-reduced pixel value of each pixel.
[0129] In the apparatus of this invention, the skin color detection module 602 is further used for:
[0130] A binary face image is generated based on the skin color region, and the pixels in the skin color region are assigned a first preset pixel value, and the pixels in other regions are assigned a second preset pixel value.
[0131] The binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, and the largest connected region is taken as the connected skin color region.
[0132] In the apparatus of this invention, the skin color detection module 602 is further used for:
[0133] Calculate the number of first pixels in the skin color region. If the number of first pixels is less than a preset threshold for the number of pixels in the skin color region, determine that the original face image has not passed the review, and issue a command to re-acquire the face image to the face acquisition interface.
[0134] The apparatus of the present invention further includes an edge detection module, used for:
[0135] The single-pixel edge of the skin-colored region is drawn using an edge detection algorithm, and the edge pixels are assigned a third preset pixel value, while the other pixels are assigned a second preset pixel value.
[0136] Calculate the number of second pixels at the edge of the single pixel. If the number of second pixels is less than a preset threshold for the number of edge pixels, determine that the original face image has not passed the review and issue a command to re-acquire the face image to the face acquisition interface.
[0137] Furthermore, the specific implementation details of the device described in the embodiments of the present invention have been described in detail in the above-described method, so the details will not be repeated here.
[0138] Figure 7 An exemplary system architecture 700 to which embodiments of the present invention can be applied is shown, including terminal devices 701, 702, 703, network 704, and server 705 (this is merely an example).
[0139] Terminal devices 701, 702, and 703 can be various electronic devices with displays and support for web browsing, and have various communication client applications installed. Users can use terminal devices 701, 702, and 703 to interact with server 705 through network 704 to receive or send messages, etc.
[0140] Network 704 is a medium used to provide a communication link between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.
[0141] Server 705 can be a server that provides various services. It should be noted that the method provided in the embodiments of the present invention is generally executed by server 705, and correspondingly, the device is generally set in server 705.
[0142] It should be understood that Figure 7 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0143] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing a terminal device of the present invention. Figure 8 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0144] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0145] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.
[0146] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this invention.
[0147] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0149] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a face image preprocessing module, a skin color detection module, and a face detection module. The names of these modules do not necessarily limit the module itself; for example, the face detection module may also be described as a "skin color region detection module."
[0150] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to perform any of the face recognition processing methods described above.
[0151] The computer program product of the present invention includes a computer program that, when executed by a processor, implements the face recognition processing method in the embodiments of the present invention.
[0152] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A face recognition processing method, characterized in that, include: Receive raw face images from the face capture interface, and perform an evaluation of the raw face images before sending them to the face database: Using the top left corner of the original face image as the origin, a rectangle is used to select the face region in the original face image to obtain the region of interest; For each pixel in the original face image, the original pixel value and weight are determined, and the pixel value and distance of each other pixel in the neighborhood are determined. According to the preset correspondence between distance and weight, the weight corresponding to each distance is determined, and then the pixel values of each pixel and all other pixels in the neighborhood are weighted and summed to obtain the weighted sum result. The weights of each pixel and all other pixels in its neighborhood are summed. The quotient of the weighted summation result and the sum of the weights is calculated. The quotient is then rounded down to obtain the noise-reduced pixel value of each pixel. The region of interest is input into a preset color model to obtain the hue and brightness values of each pixel in the region of interest; A set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range is determined to obtain the skin-colored region in the region of interest. The skin-colored region is then reviewed, including calculating the number of first pixels in the skin-colored region. If the number of first pixels is less than a preset threshold for the number of pixels in the skin-colored region, the original face image is determined to have failed the review, and a command to re-acquire the face image is sent to the face acquisition interface. The skin-colored region is then drawn using an edge detection algorithm, and edge pixels are assigned a third preset pixel value, while other pixels are assigned a second preset pixel value. Calculate the number of second pixels at the edge of the single pixel. If the number of second pixels is less than a preset threshold for the number of edge pixels, determine that the original face image has not passed the review and issue a command to re-acquire the face image to the face acquisition interface. In response to the skin color region passing the review, the preset face detection model Adaboost is invoked to identify whether the face in the original face image is complete. In response to the recognition result being complete, the original face image is encoded according to a preset fixed format and transmitted to the face database for processing through the face recognition interface.
2. The method according to claim 1, characterized in that, Before inputting the original face image into the preset color model, the method further includes: The original face image is subjected to Gaussian noise reduction to obtain a denoised face image.
3. The method according to claim 1, characterized in that, After obtaining the skin color region in the original face image, the method further includes: A binary face image is generated based on the skin color region, and the pixels in the skin color region are assigned a first preset pixel value, and the pixels in other regions are assigned a second preset pixel value. The binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, and the largest connected region is taken as the connected skin color region.
4. A face recognition processing device, characterized in that, include: The face image preprocessing module receives the original face image acquired by the face acquisition interface. Before sending the original face image to the face database, it performs an evaluation of the original face image: using the upper left corner of the original face image as the origin, a rectangle is used to select the face region in the original face image to obtain the region of interest; for each pixel in the original face image, the original pixel value and weight are determined, and the pixel value of each other pixel in the neighborhood and the distance to each pixel are determined; according to the preset correspondence between distance and weight, the weight corresponding to each distance is determined, and then the pixel values of each pixel and all other pixels in the neighborhood are weighted and summed to obtain the weighted sum result; The weights of each pixel and all other pixels in its neighborhood are summed. The quotient of the weighted summation result and the sum of the weights is calculated. The quotient is then rounded down to obtain the noise-reduced pixel value of each pixel. The region of interest is input into a preset color model to obtain the hue and brightness values of each pixel in the region of interest; The skin color detection module is used to determine the set of pixels whose hue values are within a preset hue range and whose brightness values are within a preset brightness range, so as to obtain the skin color region in the region of interest. The skin color region is then reviewed, including calculating the number of first pixels in the skin color region. In response to the number of first pixels being less than a preset threshold for the number of pixels in the skin color region, the module determines that the original face image has not passed the review and sends a command to re-acquire the face image to the face acquisition interface. The edge detection module is used to draw the single-pixel edge of the skin color region using an edge detection algorithm, assign the edge pixels to a third preset pixel value, and assign the other pixels to a second preset pixel value; Calculate the number of second pixels at the edge of the single pixel. If the number of second pixels is less than a preset threshold for the number of edge pixels, determine that the original face image has not passed the review and issue a command to re-acquire the face image to the face acquisition interface. The face detection module is used to respond to the skin color region passing the review, call the preset face detection model Adaboost to identify whether the face in the original face image is complete, and respond to the recognition result being complete, encode the original face image according to the preset fixed format and transmit it to the face database for processing through the face recognition interface.
5. The apparatus according to claim 4, characterized in that, The face image preprocessing module is also used for: The original face image is subjected to Gaussian noise reduction to obtain a denoised face image.
6. The apparatus according to claim 4, characterized in that, The skin color detection module is also used for: A binary face image is generated based on the skin color region, and the pixels in the skin color region are assigned a first preset pixel value, and the pixels in other regions are assigned a second preset pixel value. The binary face image is subjected to erosion and dilation processing to perform connectivity processing on the face, and the largest connected region is taken as the connected skin color region.
7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-3.
8. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-3.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.