Electronic face photo acquisition method based on key point and visual salient object detection
Through key points and visually significant object detection, multiple requirements are met in the process of electronic face photo acquisition, solving the problem of insufficient collection efficiency and automation in the existing technology, and providing an efficient and stable electronic face photo acquisition method .
Patent Information
- Application Number
- CN202210820790.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-12
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-07-12
AI Technical Summary
The prior art is difficult to meet multiple requirements such as face area, photo quality, size, proportion and color during the electronic face photo collection process, resulting in insufficient collection efficiency and automation.
Using a method based on key points and visual significant object detection, face uniqueness, integrity and image quality are checked. Through facial correction, significant object detection and image segmentation, image size and background color are adjusted to achieve end-to-end electronic faces. Photo collection and processing.
It improves the efficiency and stability of electronic face photo acquisition, ensures that the output image meets preset requirements, reduces manual post-processing, and is suitable for electronic face photo acquisition scenarios for large-scale populations.
Smart Images

Figure CN114973384B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of image processing, and in particular to an electronic face photo acquisition method based on key point and visual salient target detection. Background Art
[0002] As the Internet develops rapidly, the degree of social informatization is increasing. More and more application scenarios require the provision of electronic ID photos containing portraits. This puts forward new requirements for the collection of electronic face photos. It is hoped that the real-time and automation of photo shooting and processing will be further improved. For example, in large-scale electronic photo collection scenarios such as student information collection, on the one hand, the shooting system is required to be able to judge in real time whether the parameters of the taken photos meet the collection requirements. If not, it can be re-collected immediately; on the other hand, the system is required to be able to automatically process photos to avoid wasting too much manpower and time. Based on this, the demand for automated and efficient photo collection systems in various application scenarios has become increasingly prominent, and intelligent portrait photo collection and processing algorithms have also become one of the research hotspots in the field of image processing and computer vision.
[0003] As a research focus in the field of computer vision, face recognition algorithms have developed rapidly and are widely used. They can also be applied to the collection and automatic processing of electronic face photos. However, in addition to the requirements for the face area, electronic face photos also need to consider many factors such as photo quality, size, proportion, color, etc., which cannot be met by relying solely on face recognition algorithms. Considering this issue, it is necessary to design a complete algorithm process and system so that the automatic collection and processing of face photos can be completed according to different parameters. Summary of the invention
[0004] The purpose of the present invention is to provide an electronic face photo acquisition method based on key point and visual salient target detection, which can effectively extract portrait targets and realize background color replacement, realize end-to-end processing of input images, and efficiently obtain electronic face photos that meet preset requirements.
[0005] To achieve the above object, the present invention provides the following solutions:
[0006] The electronic face photo acquisition method based on key point and visual salient object detection includes:
[0007] For the input image Conducting face uniqueness inspection, face integrity inspection and image quality inspection in sequence, and determining an initial image that meets the requirements for producing an electronic photo based on the inspection results;
[0008] Performing facial correction on the initial image to obtain a facial image with a standard posture;
[0009] Performing salient object detection on the facial image of the standard posture, and performing image segmentation according to the detection result to obtain a person image with a foreground and a background of the person separated;
[0010] The character image is post-processed to adjust the image size and background color to obtain an electronic character photo that meets the requirements.
[0011] Preferably, the uniqueness check of the face comprises: Perform face detection and obtain the selection box set of the face area {f m |m=1,2,...,M}, if M=1, the uniqueness test of the face passes; if M≠1, the uniqueness test of the face fails.
[0012] Preferably, the face integrity check includes: performing key point detection on the only valid face f1 to obtain 81 key points {(x i ,y i )|i=1,2,...81},determine whether
[0013]
[0014] Among them, W is the horizontal resolution of the image, and H is the vertical resolution. If they can be met, the integrity check of the face passes. If they cannot be met, the integrity check of the face fails.
[0015] Preferably, the inspection of the image quality comprises: converting the input image into a two-dimensional grayscale image; performing a convolution operation on the grayscale image using a Laplace operator L, and calculating an image clarity index q:
[0016]
[0017] q=std(C L (x,y)) (3)
[0018] Among them, C L (x, y) is the convolution of the Laplacian operator at all pixels (x, y) of the grayscale image, std() is the standard deviation calculation; set the clarity threshold T q , if q>T q , then the input image Quality inspection passed, if q<T q , then the input image Quality inspection failed.
[0019] Preferably, the step of acquiring the facial image in the standard posture comprises: performing an affine transformation on the initial image based on an affine transformation matrix to acquire the facial image in the standard posture:
[0020]
[0021] Among them, (x, y) (x′, y′) are the input images respectively. And the image after affine transformation The pixel coordinates of the input image. Parameters a1, a2, a3, and a4 are determined by the five key points in the input image. and facial images in standard poses The coordinates (x k ,y k ), (x k′ ,y k′ ) is substituted into the solution to obtain.
[0022] Preferably, obtaining a person image with a person foreground and a person background separated comprises: segmenting the image into superpixels using a simple linear iterative clustering superpixel segmentation algorithm; constructing a graph model G(N, E) with superpixels as nodes, wherein N is a node and E is an undirected edge between nodes;
[0023] The weight matrix of the computational graph model G is Ω = {ω ij |i, j = 1, 2, ..., n} and the degree matrix D = diag{d 11 , ..., d nn},
[0024]
[0025] d ii =∑ j ω ij (6)
[0026] Among them, c i and c j is the color average of graph nodes i and j in the LAB color space, and σ is a constant;
[0027] Based on Ω and D, calculate the optimal neighbor matrix A = (D-αΩ) -1 , and calculate the ranking score r * As the saliency value of the superpixel, r * =AY, where Y = {Y i |i=1,2,...,n], is an indicator vector, indicating whether each node is a seed point. If so, Y i =1 if not, 0;
[0028] The superpixels at the upper, lower, left, and right edges of the image are used as background seed points to obtain the initial saliency map S bq , S bq (i) = S t (i)×S b (i)×Sl (i)×S r (i), i=1, 2,...,n, where, S t (i), S b (i), S l (i), S r (i) The saliency map is obtained by using the four edges of top, bottom, left, and right as seed points. is the corresponding standardized significance value vector;
[0029] Based on the initial saliency map, reselect the foreground seed points and calculate the final saliency map.
[0030] The final saliency map is tested to obtain two parts with large grayscale value differences, and the segmentation threshold T is set. seg , gray value is greater than T seg The portrait area is the grayscale value less than Tseg, and the background area is the grayscale value less than Tseg. The portrait area coordinate sets are obtained respectively. and background area coordinate set u=1, 2,..., U, v=1, 2,..., V, U+V=W×H.
[0031] Preferably, obtaining a person image with a person foreground and a background separated also includes: firstly performing salient target detection on the facial image of the standard posture, and performing image segmentation of the person foreground and the background based on a saliency threshold based on the detection result.
[0032] Preferably, adjusting the background color of the image includes: replacing the RGB three-channel color vector of the pixel point in the background area of the image, and keeping the pixel point in the foreground character area unchanged, to obtain an image with the replaced background color
[0033] Preferably, adjusting the image size includes: obtaining the highest point (x t ,y t ), the lowest point (x b ,y b ), the leftmost point (x l ,y l ), the rightmost point (x r ,y r ), the rectangular area obtained by connecting the edges of the four points is taken as the facial area, the preset width and height of the output image are W′ and H′ respectively, and the coordinates of the cropping frame corners are calculated by equations (10)-(13),
[0034]
[0035]
[0036]
[0037]
[0038] Among them, eW, eT, and eB are the distances from the facial area to the left and right edges and the upper and lower edges of the output image, respectively.
[0039]
[0040]
[0041]
[0042] The calculation method of eW ensures that the face area is centered in the output portrait photo. t , ratio b Respectively represent the relative distance ratio of the facial area to the top and bottom, ratio t and ratio b The value range is [0, 1], ratio t +ratio b =1, so eT and eB determine the distance between the top of the forehead and the upper edge of the output portrait photo, and the distance between the chin and the lower edge. The rectangular area within the coordinate range is the cropping frame to crop the image, and the output image is an electronic face photo that meets the preset requirements.
[0043] The electronic face photo collection method based on key point and visual salient target detection provided by the present invention fully considers the requirements at different levels through the image availability judgment process, designs three verification processes of face uniqueness and integrity inspection, and image clarity evaluation, and can more comprehensively judge the availability of the input image before image processing, on the one hand avoiding the waste of computing time and resources caused by processing invalid images, and on the other hand ensuring that the processed output image is valid and available, with higher efficiency and stronger stability. At the same time, the present invention adopts an end-to-end design concept, takes the original image taken by the camera as input, and automatically outputs electronic face photos that meet the preset background and size, avoiding additional manual post-processing, and can be better applied to the electronic face photo collection scenario of a large population. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1This is a flow chart of collecting electronic face photos provided in an embodiment of the present invention; DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] The purpose of the present invention is to provide an electronic face photo acquisition method based on key point and visual salient target detection, which can effectively extract portrait targets and realize background color replacement, realize end-to-end processing of input images, and efficiently obtain electronic face photos that meet preset requirements.
[0048] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, the electronic face photo acquisition method based on key point and visual salient target detection is characterized by comprising:
[0049] Performing face uniqueness check, face integrity check and image quality check on the input image in sequence, and determining an initial image that meets the requirements for producing an electronic photo based on the test results;
[0050] Performing facial correction on the initial image to obtain a facial image with a standard posture;
[0051] Performing salient object detection on the facial image of the standard posture, and performing image segmentation according to the detection result to obtain a person image with a foreground and a background of the person separated;
[0052] The character image is post-processed to adjust the image size and background color to obtain an electronic character photo that meets the requirements.
[0053] Furthermore, the uniqueness test of the face includes: obtaining a selection box set of the face area {f m |m=1,2,...,M}, if M=1, the uniqueness test of the face passes; if M≠1, the uniqueness test of the face fails.
[0054] Furthermore, the integrity check of the face includes: performing key point detection on the face to obtain 81 key points {(x i ,y i)|i=1,2,...81},determine whether max(x)<W,min(x)>0,max(y)<H,min(y)>0 are satisfied at the same time, wherein W is the horizontal resolution of the image and H is the vertical resolution. If so, the integrity check of the face passes; if not, the integrity check of the face fails;
[0055] Furthermore, the inspection of the image quality includes: converting the input image into a two-dimensional grayscale image; performing a convolution operation on the grayscale image using a Laplace operator L, and calculating an image clarity index q: q=std(C L (x, y)), where C L (x, y) is the convolution of the Laplacian operator at all pixels (x, y) of the grayscale image, std is the standard deviation calculation; set the clarity threshold T q , if q>T q , then the image quality inspection passes, if q<T q , the image quality inspection fails.
[0056] Furthermore, the step of acquiring a facial image in a standard posture includes: performing an affine transformation on the input image based on an affine transformation matrix to acquire a facially balanced image.
[0057] Furthermore, obtaining a person image with a person foreground and a person background separated includes: segmenting the image into superpixels using a simple linear iterative clustering superpixel segmentation algorithm; constructing a graph model G(N, E) with superpixels as nodes, wherein N is a node and E is an undirected edge between nodes;
[0058] The weight matrix of the computational graph model G is Ω = {ω ij |i, j = 1, 2, ..., n} and the degree matrix D = diag{d 11 , ..., d nn}, Among them, c i and c j is the color average of graph nodes i and j in the LAB color space, and σ is a constant;
[0059] Based on Ω and D, calculate the optimal neighbor matrix A = (D-αΩ) -1 , and calculate the ranking score r * As the saliency value of the superpixel, r * =AY, where Y = {Y i |i=1,2,...,n], is an indicator vector, indicating whether each node is a seed point. If so, Y i =1 if not, 0;
[0060] The superpixels at the upper, lower, left, and right edges of the image are used as background seed points to obtain the initial saliency map S bq , S bq (i) = S t (i)×S b (i)×S l (i)×S r (i), i=1, 2,...,n, where, S t (i), S b (i), S l (i), S r (i) The saliency map is obtained by using the four edges of top, bottom, left, and right as seed points. is the corresponding standardized significance value vector;
[0061] Based on the initial saliency map, reselect the foreground seed points and calculate the final saliency map.
[0062] The final saliency map is tested to obtain two parts with large grayscale value differences, and the segmentation threshold T is set. seg , gray value is greater than T seg The portrait area has a gray value less than T seg For the background area, obtain the portrait area coordinate set respectively and background area coordinate set u=1, 2,..., U, v=1, 2,..., V, U+V=W×H.
[0063] Furthermore, obtaining a person image with a person foreground and a background separated also includes: firstly performing salient target detection on the facial image of the standard posture, and performing image segmentation of the person foreground and the background based on a saliency threshold based on the detection result.
[0064] Furthermore, adjusting the background color of the image includes: replacing the RGB three-channel color vector of the pixel points in the background area of the image, and keeping the pixel points in the foreground character area unchanged, to obtain an image with the replaced background color.
[0065] Further, adjusting the image size includes: obtaining the highest point (x t ,y t ), the lowest point (x b ,y b ), the leftmost point (x l ,y l ), the rightmost point (x r ,y r ), the rectangular area obtained by connecting the edges of the four points is taken as the facial area, the preset width and height of the output image are W′ and H′ respectively, and the coordinates of the cropping box corners are calculated. Among them, eW, eT, and eB are the distances from the facial area to the left and right edges and the upper and lower edges of the output image, respectively.
[0066]
[0067] The calculation method of eW ensures that the face area is centered in the output portrait photo. t , ratio b Respectively represent the relative distance ratio of the facial area to the top and bottom, ratio t and ratio b The value range is [0, 1], ratio t +ratio b =1, so eT and eB determine the distance between the top of the forehead and the upper edge of the output portrait photo, and the distance between the chin and the lower edge. The rectangular area within the coordinate range is the cropping frame to crop the image, and the output image is an electronic face photo that meets the preset requirements.
[0068] The present invention also provides a specific embodiment step process:
[0069] Step 1.1: Face uniqueness and integrity check. Read the original image taken by the camera, perform face and face key point detection on it, and judge the uniqueness and integrity of the face in the image based on the results. If it contains a unique and complete face, it is considered to meet the production requirements and proceed to step 1.2. Otherwise, it is considered to not meet the requirements, prompt a question, require reshooting, and re-execute step 1.1.
[0070] Step 1.2, face image quality evaluation. Perform a quality evaluation on the images that pass the face uniqueness and integrity test. If they meet the quality requirements, proceed to step 2. Otherwise, mark them as not meeting the production requirements, prompt a problem, require reshooting, and re-execute step 1.1.
[0071] The image processing stage processes the input image that meets the production requirements and generates an electronic face photo according to the preset requirements, including the following steps:
[0072] Step 2.1, face correction: Based on the face detection and key point detection results, face angle correction is performed to convert the image into a standard posture without facial tilt.
[0073] Step 2.2: Perform salient target detection on the obtained standard pose face image, and perform image segmentation based on the detection result to separate the foreground and background of the person.
[0074] Step 2.3, perform image post-processing, adjust the image size, background color, etc. according to preset parameters, and obtain an electronic face photo that meets the preset requirements.
[0075] Furthermore, the uniqueness and integrity of the face in step 1.1, where uniqueness means that there is only one valid face in the input image; integrity means that the detected face is complete and contains all the key elements of the face. The inspection is achieved by means of face detection and face key point detection, and the specific process is as follows:
[0076] Step 1.1.1, input image Perform face detection and obtain a selection box set containing the face area {f m |m=1,2,...,M}, and based on this, the execution conditional judgment is:
[0077] (1) If M = 1, it means that only one valid face is detected in the image. The image is marked as meeting the requirements and the process goes to step 1.1.2.
[0078] (2) If M>1, it means that more than one valid face is detected in the image. The image is marked as not meeting the uniqueness requirement and the subsequent steps are not performed. The image is prompted to be re-photographed and the current step is performed again.
[0079] (3) If M=0, it indicates that no valid face is detected in the image, and the image is marked as not meeting the uniqueness requirement. The subsequent steps are not performed, and the image is prompted to be re-photographed and the current step is re-executed.
[0080] Step 1.1.2: perform key point detection on the only valid face f1 detected in step 1.1.1, and obtain 81 facial key points ranging from forehead to chin, including eyebrows, eyes, nose, and mouth contours {(x i ,y i )|i=1,2,...81},determine whether the face in the image is complete by the coordinates of the key points of the facial contour. Assume that the horizontal resolution of the image is W and the vertical resolution is H. If the key point coordinates meet the following conditions, it means that the facial contour is completely contained in the image, marked as meeting the integrity requirements, and proceed to step 1.2:
[0081]
[0082] If any of the above conditions is not met, it means that part of the face is not included in the image, the face is not complete, and it is marked as not meeting the integrity requirements. No further steps will be performed, and a prompt will be given to re-shoot and re-execute step 1.1. It should be noted that if the key facial organs are missing, a valid face cannot be detected, that is, step 1.1.1 cannot be passed. Therefore, only the facial contour integrity is tested here, and the key facial organ integrity is not tested again.
[0083] Furthermore, the facial image quality evaluation in step 1.2 uses the Laplace gradient function to evaluate the image clarity, including the following steps:
[0084] Step 1.2.1, for the input image It is converted into a two-dimensional grayscale image, and the grayscale image is convolved using the Laplace operator L as shown in formula (2) to calculate the image clarity index q:
[0085]
[0086] q=std(C L (x,y)) (3)
[0087] Among them, C L (x, y) is the convolution of the Laplacian operator at all pixels (x, y) of the grayscale image, and std() is the standard deviation calculation.
[0088] Step 1.2.2: Determine whether the image quality meets the requirements based on the calculated clarity index q. Set the clarity threshold T q , if q<T q , it indicates that the image clarity does not meet the requirements, and the subsequent steps are not performed. It is prompted to adjust the camera focus and other parameters, reshoot and execute step 1.1; if q>T q , it means that the image meets the clarity requirements and proceed to step 2.
[0089] Furthermore, the facial correction described in step 2.1 aims to transform the image into a standard posture without facial tilt, and the specific implementation process is as follows:
[0090] Step 2.1.1, from the 81 facial key points in step 1.1.2, take the coordinates of the two end points of the left and right eye corners and the end point of the nose, a total of 5 key points {(x k ,y k )|k=1, 2, 3, 4, 5} as the reference point for image transformation.
[0091] Step 2.1.2, transform the image based on the following affine transformation matrix Perform an affine transformation to convert the face into a standard posture with balanced face:
[0092]
[0093] Among them, (xy)(x′, y′) are the input images respectively. And the image after affine transformation The pixel coordinates of the input image. Parameters a1, a2, a3, and a4 are determined by the five key points in the input image. And standard posture image The coordinates (x k ,y k ), (x k′ ,yk′ ) is substituted into the solution to obtain.
[0094] Furthermore, the salient object detection in step 2.2 refers to using a graph-based popular ranking method to detect the image containing the standard pose face in step 2.1. Perform salient target detection and perform image segmentation based on saliency threshold based on the detection results to separate the foreground and background of the person. The specific implementation method is as follows:
[0095] Step 2.2.1, image The image is segmented into superpixels using a simple linear iterative clustering superpixel segmentation algorithm, and a graph model G(N, E) is constructed with superpixels as nodes, where N is a node and E is an undirected edge between nodes.
[0096] Step 2.2.2: Calculate the weight matrix of the graph model G containing n nodes
[0097] Ω={ω ij |i, j = 1, 2, ..., n} and the degree matrix D = diag{d 11 , ..., d nn}:
[0098]
[0099] d ii =∑ j ω ij (6)
[0100] Among them, c i and c j is the average color value of graph nodes i and j in the LAB color space, and σ is a constant.
[0101] Step 2.2.3, based on Ω and D, calculate the optimal neighbor matrix A = (D-αΩ) -1 , and calculate the ranking score r * As the saliency value of the superpixel:
[0102] r * =AY (7)
[0103] Where Y = {Y i |i=1,2,...,n] is an indicator vector indicating whether each node is a seed point. If so, Y i =1, otherwise 0. The superpixels at the four edges of the image are used as background seed points to obtain the initial saliency map S bq :
[0104] S bq (i) = S t (i)×S b(i)×S l (i)×S r (i), i=1, 2, ..., n (8)
[0105] Among them, S t (i), S b (i), S l (i), S r (i) is the saliency map obtained by taking the four edges of top, bottom, left and right as seed points. is the corresponding normalized significance value vector.
[0106] Step 2.2.4, based on the initial saliency map, reselect the foreground seed point and calculate the final saliency map:
[0107]
[0108] Step 2.2.5: After salient target detection, a saliency map is obtained. The foreground target has a clear saliency difference with the background. The grayscale value of the foreground area is close to 255, while the grayscale value of the background area is close to 0. Therefore, the segmentation threshold T is set. seg , the gray value is greater than T seg The portrait area is smaller than T seg is divided into background areas, and the portrait area coordinate sets are obtained respectively and background area coordinate set
[0109] u=1, 2,..., U, v=1, 2,..., V, U+V=W×H;
[0110] Furthermore, the image post-processing described in step 2.3 is to process the image according to the preset parameters. The size, background color, etc. are adjusted to obtain an electronic face photo that meets the preset requirements. The specific implementation process is as follows:
[0111] Step 2.3.1, image The median coordinate is The RGB three-channel color vector c of the background area pixel point is replaced with the preset color vector c′, and the foreground portrait area remains unchanged to obtain an image with replaced background color
[0112] Step 2.3.2, replace the background color of the image Crop to the preset size. Take the highest point (x t ,y t ), the lowest point (x b ,y b ), the leftmost point (x l ,yl ), the rightmost point (x r ,y r ), the rectangular area obtained by connecting the edges of the above four points is taken as the facial area, that is, the endpoints are (x l ,y t ), (x l ,y b ), (x r ,y t ), (x r ,y b ). Assuming that the preset width and height of the output image are W′ and H′ respectively, the coordinates of the cropping box corners are calculated by equations (10)-(13):
[0113]
[0114]
[0115]
[0116]
[0117] Among them, eW, eT, and eB are the distances from the facial area to the left and right edges and the upper and lower edges of the output image respectively:
[0118]
[0119]
[0120]
[0121] The calculation method of eW ensures that the face area is centered in the output portrait photo. t , ratio b Respectively represent the relative distance ratio of the facial area to the top and bottom, ratio t and ratio b The value range is [0, 1], ratio t +ratio b =1, so eT and eB determine the distance between the top of the forehead and the upper edge of the output portrait photo, and the distance between the chin and the lower edge. The rectangular area within the above coordinate range is used as the cropping frame for the image. After cropping, the output image is an electronic face photo that meets the preset requirements.
[0122] The electronic face photo collection method based on key point and visual salient target detection provided by the present invention fully considers the requirements at different levels through the image availability judgment process, designs three verification processes of face uniqueness and integrity inspection, and image clarity evaluation, and can more comprehensively judge the availability of the input image before image processing, on the one hand avoiding the waste of computing time and resources caused by processing invalid images, and on the other hand ensuring that the processed output image is valid and available, with higher efficiency and stronger stability. At the same time, the present invention adopts an end-to-end design concept, takes the original image taken by the camera as input, and automatically outputs electronic face photos that meet the preset background and size, avoiding additional manual post-processing, and can be better applied to the electronic face photo collection scenario of a large population.
[0123] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for collecting electronic face photos based on key point and visual salient object detection, characterized in that: include: For the input image Conducting face uniqueness inspection, face integrity inspection and image quality inspection in sequence, and determining an initial image that meets the requirements for producing an electronic photo based on the inspection results; Performing facial correction on the initial image to obtain a facial image with a standard posture; Performing salient object detection on the facial image of the standard posture, and performing image segmentation according to the detection result to obtain a person image with a foreground and a background of the person separated; The character image is post-processed to adjust the image size and background color to obtain an electronic character photo that meets the requirements.
2. The electronic face photo collection method according to claim 1, characterized in that: The uniqueness test of the face includes: Perform face detection and obtain the selection box set of the face area {f m |m=1,2,...,M}, if M=1, the uniqueness test of the face passes; if M≠1, the uniqueness test of the face fails.
3. The electronic face photo collection method according to claim 2, characterized in that: The face integrity check includes: performing key point detection on the only valid face f1, obtaining 81 key points {(x i ,y i )|i=1,2,...81},determine whether Among them, W is the horizontal resolution of the image, and H is the vertical resolution. If they can be met, the integrity check of the face passes. If they cannot be met, the integrity check of the face fails.
4. The electronic face photo collection method according to claim 3, characterized in that: The image quality inspection includes: converting the input image into a two-dimensional grayscale image; performing a convolution operation on the grayscale image using a Laplace operator L, and calculating an image clarity index q: q=std(C L (x,y)) (3) Among them, C L (x, y) is the convolution of the Laplacian operator at all pixels (x, y) of the grayscale image, std() is the standard deviation calculation; set the clarity threshold T q , if q>T q , then the input image Quality inspection passed, if q<T q , then the input image Quality inspection failed.
5. The electronic face photo collection method according to claim 1, characterized in that: The step of acquiring the facial image in the standard posture comprises: performing an affine transformation on the initial image based on an affine transformation matrix to acquire the facial image in the standard posture: Among them, (x, y) (x′, y′) are the input images respectively. And the image after affine transformation The pixel coordinates, parameters a1, a2, a3, a4 are determined by 5 key points in the input image and facial images in standard poses The coordinates (x k ,y k ), (x k′ ,y k′ ) is substituted into the solution to obtain.
6. The electronic face photo collection method according to claim 1, characterized in that: Obtaining a person image with a person foreground and a person background separated includes: using a simple linear iterative clustering superpixel segmentation algorithm to segment the image into superpixels; constructing a graph model G(N, E) with superpixels as nodes, wherein N is a node and E is an undirected edge between nodes; The weight matrix of the computational graph model G is Ω = {ω ij |i, j = 1, 2, ..., n} and the degree matrix D = diag{d 11 , ..., d nn }, d ii =∑ j oh ij (6) Among them, c i and c j is the color average of graph nodes i and j in the LAB color space, and σ is a constant; Based on Ω and D, calculate the optimal neighbor matrix A = (D-αΩ) -1 , and calculate the ranking score r * As the saliency value of the superpixel, r * =AY, where Y = {Y i |i=1,2,...,n], is an indicator vector, indicating whether each node is a seed point. If so, Y i =1, otherwise 0; The superpixels at the upper, lower, left, and right edges of the image are used as background seed points to obtain the initial saliency map S bq , S bq (i) = S t (i)×S b (i)×S l (i)×S r (i), i=1, 2,...,n, where, S t (i), S b (i), S l (i), S r (i) The saliency map is obtained by using the four edges of top, bottom, left, and right as seed points. is the corresponding standardized significance value vector; Based on the initial saliency map, reselect the foreground seed points and calculate the final saliency map. The final saliency map is tested to obtain two parts with large grayscale value differences, and the segmentation threshold T is set. seg , gray value is greater than T seg The portrait area has a gray value less than T seg For the background area, obtain the portrait area coordinate set respectively and background area coordinate set u=1, 2,..., U, v=1, 2,..., V, U+V=W×H.
7. The electronic face photo collection method according to claim 6, characterized in that: Acquiring a person image with a person foreground and a background separated also includes: firstly performing salient target detection on the facial image of the standard posture, and performing image segmentation of the person foreground and the background based on a saliency threshold based on the detection result.
8. The electronic face photo collection method according to claim 1, characterized in that: Adjusting the background color of an image includes: replacing the RGB three-channel color vector of the pixel points in the background area of the image, while keeping the pixel points in the foreground character area unchanged, and obtaining an image with the replaced background color 9. The electronic face photo collection method according to claim 1, characterized in that: Adjusting the image size includes: taking the highest point among the facial key points (x t ,y t ), the lowest point (x b ,y b ), the leftmost point (x1, y1), the rightmost point (x r ,y r ), the rectangular area obtained by connecting the edges of the four points is taken as the facial area, the preset width and height of the output image are W′ and H′ respectively, and the coordinates of the cropping frame corners are calculated by equations (10)-(13), Among them, eW, eT, and eB are the distances from the facial area to the left and right edges and the upper and lower edges of the output image, respectively. The calculation method of eW ensures that the face area is centered in the output portrait photo, where ratio t , ratio b Respectively represent the relative distance ratio of the facial area to the top and bottom, ratio t and ratio b The value range is [0, 1], ratio t +ratio b =1, so eT and eB determine the distance between the top of the forehead and the upper edge of the output portrait photo, and the distance between the chin and the lower edge. The rectangular area within the coordinate range is the cropping frame to crop the image, and the output image is an electronic face photo that meets the preset requirements.
