Identification photo generation method and device based on face feature points and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides an identification photo generation method based on face feature points, and the method comprises the steps: detecting a face region of a source image; extracting face feature points from the face region of the source image to obtain coordinates of the face feature points; calculating translation deviation and angle deviation of the face in the source image relative to the face in the standard identification photo according to the coordinates of the face feature points; based on the translation deviation and the angle deviation, performing posture correction on the face in the source image through matrix transformation and scale adjustment so as to align the face in the standard identification photo; and cutting the source image after face alignment to generate a target identification photo. According to the technical scheme, the precision and efficiency of aligning the face of the source image and the face posture of the standard identification photo can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and storage medium for generating an ID photo based on facial feature points. Background Art

[0002] ID photos refer to photos used on various ID cards (such as ID cards, passports, driver's licenses, student ID cards, etc.). Since ID photos are an important basis for proving identity, they must be able to clearly show personal features, and strict shooting standards and specifications can reduce the risk of forgery and abuse of ID cards and maintain social security. Therefore, ID photos usually need to meet certain standards and requirements to ensure that individuals can be accurately and clearly identified when verifying their identities. Generally speaking, when taking ID photos, users will adjust the images taken for the first time to meet the standards of ID photos. However, this method of manually adjusting the image by manually positioning the head and adjusting the posture is prone to errors, especially for images with complex backgrounds or shooting angles. This manual adjustment method is less effective and less efficient. Summary of the invention

[0003] The present application provides a method, device and storage medium for generating ID photos based on facial feature points, which can improve the accuracy and efficiency of aligning the face of a source image with the facial posture of a standard ID photo.

[0004] On the one hand, the present application provides a method for generating a ID photo based on facial feature points, the method comprising:

[0005] Detect the face area of ​​the source image;

[0006] Extracting facial feature points from the face area of ​​the source image to obtain coordinates of the facial feature points;

[0007] Calculating the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points;

[0008] Based on the translation deviation and the angle deviation, performing posture correction on the face in the source image through matrix transformation and scale adjustment to align the face in the standard ID photo;

[0009] Crop the source image after face alignment to generate the target ID photo.

[0010] On the other hand, the present application provides a device for generating a ID photo based on facial feature points, the device comprising:

[0011] A detection module, used to detect the face area of ​​the source image;

[0012] An extraction module, used to extract facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points;

[0013] A calculation module, used to calculate the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points;

[0014] A correction module, configured to perform posture correction on the face in the source image through matrix transformation and scale adjustment based on the translation deviation and the angle deviation so as to align the face in the standard ID photo;

[0015] The generation module is used to crop the source image after face alignment to generate a target ID photo.

[0016] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the technical solution of the method for generating ID photos based on facial feature points as described above when executing the computer program.

[0017] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the method for generating an ID photo based on facial feature points as described above.

[0018] From the technical solution provided by the present application, it can be seen that after extracting facial feature points from the face area of ​​the source image and obtaining the coordinates of the facial feature points, the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo are calculated based on the coordinates of the facial feature points. Based on these translation deviations and angle deviations, the posture of the face in the source image is corrected through matrix transformation and scale adjustment to align the face in the standard ID photo. Finally, the source image after face alignment is cropped to generate the target ID photo. Compared with the prior art method of manually adjusting the image by manually positioning the head and adjusting the posture, the technical solution of the present application can be automatically completed through an artificial intelligence model without manual participation or intervention, thereby improving the accuracy and efficiency of the posture alignment of the face in the source image with the face in the standard ID photo, especially for source images based on complex backgrounds or shot at angles, the alignment accuracy or effect is more obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 is a flow chart of a method for generating ID photos based on facial feature points provided in an embodiment of the present application;

[0021] Figure 2 It is a structural schematic diagram of a device for generating ID photos based on facial feature points provided in an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In this specification, adjectives such as first and second may be used only to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be interpreted as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0025] In this specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0026] ID photos refer to photos used on various ID cards (such as ID cards, passports, driver's licenses, student ID cards, etc.). Since ID photos are an important basis for proving identity, they must be able to clearly show personal features, and strict shooting standards and specifications can reduce the risk of forgery and abuse of ID cards and maintain social security. Therefore, ID photos usually need to meet certain standards and requirements to ensure that individuals can be accurately and clearly identified when verifying their identities. Generally speaking, when taking ID photos, users will adjust the images taken for the first time to meet the standards of ID photos. However, this method of manually adjusting the image by manually positioning the head and adjusting the posture is prone to errors, especially for images with complex backgrounds or shooting angles. This manual adjustment method is less effective and less efficient.

[0027] In view of the above problems of the prior art, this application proposes a method for generating ID photos based on facial feature points, the flow chart of which is shown in the attached figure. Figure 1 As shown, it mainly includes steps S101 to S105, which are described in detail as follows:

[0028] Step S101: Detect the face area of ​​the source image.

[0029] In the embodiments of the present application, the source image is a photo containing a face taken by various image acquisition devices, such as a camera, and this photo is also the photo to be processed into an ID photo later. Since the ID photo mainly focuses on the face part, it is first necessary to detect the face area from the source image. From the perspective of image processing, the face area is actually a closed geometric figure, that is, the face area has a boundary or edge. Therefore, an edge detection algorithm can be used to detect the face area of ​​the source image. For example, an edge detection algorithm based on the sobel operator, canny operator or other operators can detect the face area from the source image. This application does not limit the specific algorithm.

[0030] Step S102: extracting facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points.

[0031] Facial landmarks are representative key points on the face, such as the corners of the mouth, corners of the eyes, tip of the nose, eyebrows, etc., which can all be used as facial landmarks. Facial landmarks are relatively stable, unlike the overall appearance of the face, which may be greatly affected by factors such as hairstyle and makeup. In the process of ID photo posture alignment, operations based on facial landmarks can capture the essential features of the face and reduce recognition errors caused by external interference. Precisely because facial landmarks can be used as precise reference points in ID photo posture alignment, it is necessary to extract facial landmarks from the face area of ​​the source image to obtain the coordinates of the facial landmarks. As an embodiment of the present application, extracting facial landmarks from the face area of ​​the source image to obtain the coordinates of the facial landmarks can be achieved through steps S1021 to S1024, as described in detail as follows:

[0032] Step S1021: Calculate the gradient values ​​I of the source image in the X-axis direction and the Y-axis direction of the two-dimensional coordinate system respectively. x and I y .

[0033] In the embodiment of the present application, the Sobel operator can be used to calculate the gradient values ​​I of the source image I(x, y) in the X-axis direction and the Y-axis direction of the two-dimensional coordinate system. x and I y Specifically, for any pixel point (x, y) in the source image, if the template G of the Sobel operator in the X-axis direction is taken x for The gradient value I in the X-axis direction is obtained through convolution operation x =I(x,y)*G x Similarly, if we take the template G of the Sobel operator in the X-axis direction y for The gradient value I in the Y-axis direction is obtained through convolution operation y =I(x,y)*G y In the above expression, * represents the convolution operation.

[0034] Step S1022: Use Gaussian function to and I x *I y Filtering is performed to obtain matrices A, B and C respectively.

[0035] In the embodiment of the present application, the Gaussian function is recorded as Where σ is the standard deviation, x and y are the coordinates of the pixel points. and I x *I y Filtering is actually and I x *I y Perform convolution operation, that is And C = G(x,y)*I x *I y .

[0036] Step S1023: Calculate the feature point response function R(x,y)=Det(M)-k*(trace(M)) according to matrices A, B and C. 2 The value R, where Det(M) is the determinant of M, trace(M)=A+B is the trace of M, and k is an empirical parameter.

[0037] In the embodiment of the present application, the value R of the feature point function is used to determine whether a certain point is a facial feature point. It should be noted that k determines the trade-off between the feature values, and is therefore an empirical parameter for adjusting the detection sensitivity, and is usually taken between 0.04 and 0.06. Although k is an empirical parameter, it is usually determined according to empirical rules. However, it can also be determined using an adaptive method, that is, the value of k is dynamically adjusted according to the characteristics of the source image. For example, the k value can be adjusted according to factors such as the brightness and contrast of the image. Specifically, for areas with low brightness (less than a certain threshold) in the source image, the k value can be increased, because the darker areas in the source image may require higher detection sensitivity to capture feature points. Conversely, for areas with high brightness (greater than a certain threshold) or more concentrated in the source image, the k value can be lowered, because the brighter areas in the source image do not require higher detection sensitivity to capture feature points; when the contrast of the source image is low (less than a certain threshold), the feature points may be relatively inconspicuous, and the k value can be appropriately increased to improve the detection sensitivity, and when the contrast is high (greater than the threshold), the k value can be appropriately lowered. Of course, the k value can also be adjusted by combining the information of brightness and contrast, specifically including the following steps S1 to S3:

[0038] Step S1: Determine the quantitative indexes of brightness and contrast.

[0039] Among them, for the quantitative index of brightness, the average brightness value L of the source image can be calculated. For example, for a grayscale image, Where M and N are the number of rows and columns of the source image, respectively, and I(i, j) is the pixel value of the source image at position (i, j). For the quantitative index of contrast, the standard deviation C of the source image can be calculated, that is:

[0040]

[0041] Step S2: Establish a relationship model between brightness, contrast and k value.

[0042] Specifically, first, a basic k value range can be set based on experience or preliminary experiments, such as k∈[0.04,0.06]; if L<L low (L low is the set low brightness threshold), then the k value is adjusted to k'=k×(1+α L ), where α L is the brightness adjustment factor, such as α L =0.1; if L>L high (L high is the high brightness threshold), then the k value is adjusted to k'=k×(1-β L ), where β L is the brightness adjustment coefficient, such as β C=0.1; if C<C low (C low is the set low contrast threshold), then the k value is adjusted to k'=k×(1+α C ), where α C is the contrast adjustment factor, for example, α C =0.1; if C>C high (C high is the high contrast threshold), then the k value is adjusted to k'=k×(1-β C ), where β C is the contrast adjustment factor, for example, β C =0.1.

[0043] Step S3: Comprehensively adjust the k value.

[0044] Specifically, first, a comprehensive adjustment factor F is defined, which can be a weighted sum of brightness and contrast adjustment factors, for example, F = w L ×f L +w C ×f C , where w L and w C are the weights of brightness and contrast respectively (w L +w C =1), f L and f C are the adjustment factors calculated based on brightness and contrast respectively (for example, if both brightness and contrast increase the k value, then f L and f C is a positive value; if the k value is reduced, it is a negative value); then, the k value is adjusted to k'=k×(1+F) according to the comprehensive adjustment factor F. It should be noted that in the above embodiment, the threshold L low , L high , C low , C high and the adjustment factor α L , β L , α C , β C and weight w L 、w C It can be adjusted and optimized according to the specific image type and application scenario, or the best value can be determined through a large number of experiments.

[0045] Step S1024: perform non-maximum suppression on the value R, extract facial feature points in the face area and record their coordinates.

[0046] Since the values ​​of the feature point response function are different at different points in the image, especially based on the complexity of the image, there may be some local areas where the R values ​​of multiple points are relatively high, but not all of these points should be regarded as true facial feature points. For example, in the edge area of ​​the image, the R values ​​of several adjacent points may be relatively large due to factors such as gradient changes, but in fact these points may only be related points on the edge, not true facial feature points. Non-maximum suppression can compare the R values ​​of each point with the points in its neighborhood, and only retain the points with the largest R values ​​in the neighborhood as facial feature points. In this way, false feature points generated by local changes can be filtered out, improving the accuracy of facial feature point detection. Specifically, in the embodiment of the present application, the non-maximum suppression of the value R can be performed by setting a threshold T (this threshold can be determined according to factors such as the overall grayscale range of the image and the noise level); then, the points in the image are traversed, and for each point (x, y) in the image, its response function value is R(x, y), and with the point (x, y) as the center, R(x, y) is compared with the R values ​​of other points in the neighborhood in its selected neighborhood; if R(x, y) is the largest value in the neighborhood, then the point is a facial feature point, otherwise, the point is excluded from the facial feature points. After the facial feature points are extracted into the face area, their coordinates are recorded.

[0047] Step S103: Calculate the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo based on the coordinates of the facial feature points.

[0048] Specifically, according to the coordinates of the facial feature points, the horizontal distance difference and the vertical distance difference between the center of the face in the source image and the center of the face in the standard ID photo are calculated as the translation deviation of the face in the target image relative to the face in the standard ID photo; according to the coordinates of the facial feature points, the deviation angle of rotation around the z-axis, the deviation angle of rotation around the x-axis and the deviation angle of rotation around the y-axis of the face in the source image and the face in the standard ID photo are calculated as the angular deviation of the face in the source image relative to the face in the standard ID photo. In the above embodiment, the deviation angle of rotation around the z-axis of the face in the source image and the face in the standard ID photo is the rotation angle of the face in the source image relative to the face in the standard ID photo, the deviation angle of rotation around the x-axis of the face in the image and the face in the standard ID photo is the up-down tilt angle of the face in the source image relative to the face in the standard ID photo, and the deviation angle of rotation around the y-axis of the face in the source image and the face in the standard ID photo is the left-right tilt angle of the face in the image relative to the face in the standard ID photo.

[0049] Here, it is assumed that the facial feature points detected in the source image include the left eye, the right eye, and the nose tip, where the coordinates of the left eye are marked as (x left ,y left ), the coordinates of the right eye are marked as (x right ,y right ), the coordinates of the nose tip are marked as (xnose ,y nose ); accordingly, the coordinates of the left eye in the standard ID photo are (x left_std ,y left_std ), the coordinates of the right eye are marked as (x right_std ,x left_std ), the coordinates of the nose tip are marked as (x nose_std ,y nose_std ), then according to the coordinates of the facial feature points, calculate the horizontal distance difference t between the face center in the source image and the face center in the standard ID photo x Can be The vertical distance difference t between the face center in the source image and the face center in the standard ID photo y Can be According to the coordinates of the facial feature points, calculate the deviation angle θ between the face in the source image and the face in the standard ID photo around the z-axis yaw Can be According to the coordinates of the facial feature points, calculate the deviation angle θ of the face in the source image and the face in the standard ID photo around the x-axis pitch Can be According to the coordinates of the facial feature points, calculate the deviation angle θ of the face in the source image and the face in the standard ID photo around the y-axis roll Can be

[0050] Step S104: Based on the translation deviation and the angle deviation, the posture of the face in the source image is corrected by matrix transformation and scale adjustment to align the face in the standard ID photo.

[0051] The process of correcting the posture of the face in the source image by matrix transformation and scale adjustment is also the process of correcting the parameters of the matrix transformation and the scale of the source image so that the above deviation is 0 or within a preset threshold. As an embodiment of the present application, based on the translation deviation and the angle deviation, the posture correction of the face in the source image by matrix transformation and scale adjustment to align the face in the standard ID photo can be achieved through steps S1041 to S1043, which are described in detail as follows:

[0052] Step S1041: Calculate the corresponding first rotation matrix R according to the deviation angle of rotation around the z-axis, the deviation angle of rotation around the x-axis, and the deviation angle of rotation around the y-axis between the face in the source image and the face in the standard ID photo. yaw , the second rotation matrix R pitch , the third rotation matrix R roll And the final rotation matrix R is as follows:

[0053]

[0054]

[0055]

[0056] R=R yaw ×R pitch ×R roll

[0057] Among them, θ yaw is the deviation angle between the face in the source image and the face in the standard ID photo around the z-axis, θ pitch is the deviation angle of rotation around the x-axis between the face in the source image and the face in the standard ID photo, θ roll It is the deviation angle of rotation around the y-axis between the face in the source image and the face in the standard ID photo.

[0058] Step S1042: Using the width W of the face in the standard ID photo std The width W of the face in the source image img , calculate the scale adjustment coefficient

[0059] In order to make the proportion of the face in the source image consistent with that of the standard ID photo, a scale adjustment coefficient can be calculated based on the ratio of the face width or height of the two. The scale adjustment coefficient will be used to adjust the proportion of the face in the source image during subsequent processing.

[0060] Step S1043: According to the formula Transform the source image until all deviations are corrected to below the preset threshold, where x and y are the horizontal and vertical coordinates of any pixel in the source image, respectively, and the affine transformation matrix R 2D is the two-dimensional rotation matrix part extracted from the final rotation matrix R, t x and t y They are the horizontal and vertical distance differences between the face center in the source image and the face center in the standard ID photo, respectively.

[0061] R 2D is the two-dimensional rotation matrix part extracted from the final rotation matrix R, specifically, the submatrix composed of elements extracted from the first and second rows and the first and second columns of the final rotation matrix R. When according to the formula Once all deviations are corrected to below a preset threshold, the face in the source image will be aligned with the face in the standard ID photo.

[0062] It should be noted that, in the above embodiment, according to the formula Transform the source image until all deviations are corrected to below the preset threshold, which is actually to optimize the affine transformation matrix T based on the gradient descent algorithm. affine-trans, until all deviations are corrected to be below a preset threshold, specifically, including: defining an objective function, wherein the objective function is used to measure the reprojection error between the facial feature points in the source image and the facial feature points in the standard ID photo; calculating the objective function relative to the affine transformation matrix T affine-trans The gradient of the parameters; according to the affine transformation matrix T affine-trans The gradient of the parameters of , and the step size of the update is controlled by the learning rate to update the affine transformation matrix T affine-trans Parameters of until the preset convergence condition is met. In the above embodiment, the affine transformation matrix T affine-trans The parameters include the translation deviation and angle deviation mentioned in the above embodiment, that is, the deviation angle of rotation around the z-axis, the deviation angle of rotation around the x-axis, and the deviation angle of rotation around the y-axis between the face in the source image and the face in the standard ID photo. According to the affine transformation matrix T affine-trans The gradient of the parameters of , and the step size of the update is controlled by the learning rate to update the affine transformation matrix T affine-trans The parameters of the transformation matrix T are updated in an iterative process. affine-trans The parameters will move a small step in the opposite direction of the gradient. The size of this step is controlled by the learning rate. The preset convergence condition can be that the objective function value changes less than a threshold value for several consecutive iterations, or the maximum number of iterations is reached. When the iterative process converges, the obtained affine transformation matrix can be used to transform the source image into the target image.

[0063] The learning rate in the above-mentioned gradient descent algorithm is a key parameter, which determines the step size of the parameter update at each iteration. The adjustment of the learning rate has a great influence on the convergence speed and final result of the algorithm. Although the initial learning rate is usually selected as a smaller value based on experience, such as 0.01 or 0.001, to avoid instability caused by excessive parameter updates. However, a learning rate that is too small may lead to slow convergence, and a learning rate that is too large may cause the algorithm to diverge or become unstable. Therefore, in the embodiment of the present application, the learning rate can be adaptively adjusted. In other words, in the above-mentioned embodiment, according to the affine transformation matrix T affine-trans The gradient of the parameters of , and the step size of the update is controlled by the learning rate to update the affine transformation matrix T affine-trans , until the preset convergence condition is met, it may also include: calculating the first-order momentum and the second-order momentum; according to the defined objective function relative to the affine transformation matrix T affine-trans The current gradient of the parameter is adjusted by modifying the first-order momentum and the second-order momentum. In the above embodiment, according to the formula m t =β1*m t-1 +(1-β1)*g t Calculate the first-order momentum, where m t-1 is the first-order momentum of the previous step, g tis the current gradient. The meaning of this formula is that m t is the weighted average of past gradients, where β1 determines the historical gradient information g t-1 The weight ratio in the current first-order momentum calculation, 1-β1 is the current gradient g t For example, when β1 = 0.9, it means that the current first-order momentum m t 90% depends on the first-order momentum m of the previous step t-1 , 10% depends on the current gradient g t According to the formula Calculate the second-order momentum, where v t-1 is the second-order momentum of the previous step, is the square of the current gradient. β2 determines the historical second-order momentum information v t-1 The weight ratio in the current second-order momentum calculation, 1-β2 is the current gradient square For example, when β2 = 0.999, it means that the current second-order momentum v t 99.9% depends on the second-order momentum v of the previous step t- 1, 0.1% depends on the square of the current gradient

[0064] Let's explain the objective function defined relative to the affine transformation matrix T affine-trans The current gradient of the parameters of the learning rate is adjusted by correcting the first-order momentum and the second-order momentum. Since the initial values ​​of β1 and β2 are close to \(1.0\), the deviation of the estimated time will be 0, so deviation correction is required. The corrected first-order momentum Here, t is the number of iterations, and the modified second-order momentum is Finally, update the parameters according to the above calculation results Among them, α is the learning rate, is the modified first-order momentum, is the corrected second-order momentum, and ε is a minimum value to prevent the denominator from being 0. The meaning of this formula is to adjust the parameter w by calculating the corrected first-order momentum and second-order momentum, and the set learning rate α t , realize the automatic adjustment of the learning rate of each parameter, so that the algorithm can adaptively adjust the learning speed on different parameters according to the historical information of the gradient.

[0065] It should be noted that in the parameter update formula In the above example, α is fixed, but and It is constantly changing with the iterative process. reflects the weighted average of the gradient, reflects the weighted average of the squared gradient. Different parameters in the optimization process have different gradients due to their t changes will lead to and For example, for a parameter, if its gradient changes greatly in consecutive iterations, then will be relatively large, which will make The value of is relatively small, so the update step size on this parameter is relatively small, which is equivalent to the learning rate of this parameter being "adjusted" to be smaller in actual effect. On the contrary, if the gradient of a parameter is relatively stable, Relatively small, The value of is relatively large, and the update step size of this parameter is relatively large, which is equivalent to the learning rate of this parameter being "adjusted" larger in actual effect.

[0066] Step S105: cropping the source image after face alignment to generate a target ID photo.

[0067] After the aforementioned steps S101 to S104, the center of the face has been positioned at the expected position of the source image, that is, the geometric position, rotation angle, etc. of the face in the source image are already within the standard area, which means that no additional adjustments are required during subsequent cropping. However, the source image after face alignment still needs to be cropped. The specific method can be to determine the cropping area, that is, according to the size of the standard ID photo, combined with the position and proportion of the face in the source image, determine the center point and size of the cropping area, and use the image processing library to crop the source image to ensure that the face is centered in the final ID photo and meets the standard size and position requirements, so as to obtain an ID photo that meets the standard size.

[0068] From the above attached Figure 1 It can be seen from the example of the ID photo generation method based on facial feature points that facial feature points are extracted from the face area of ​​the source image, and after the coordinates of the facial feature points are obtained, the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo are calculated based on the coordinates of the facial feature points. Based on these translation deviations and angle deviations, the posture of the face in the source image is corrected through matrix transformation and scale adjustment to align the face in the standard ID photo. Finally, the source image after face alignment is cropped to generate the target ID photo. Compared with the prior art method of manually adjusting the image by manually positioning the head and adjusting the posture, the technical solution of the present application can be automatically completed through an artificial intelligence model without manual participation or intervention, thereby improving the accuracy and efficiency of the posture alignment of the face in the source image and the face in the standard ID photo, especially for source images based on complex backgrounds or shot at angles, the alignment accuracy or effect is more obvious.

[0069] Please see attached Figure 2, is a device for generating ID photos based on facial feature points provided in an embodiment of the present application, the device may include a detection module 201, an extraction module 202, a calculation module 203, a correction module 204 and a generation module 205, which are described in detail as follows:

[0070] A detection module 201 is used to detect the face area of ​​a source image;

[0071] An extraction module 202 is used to extract facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points;

[0072] A calculation module 203 is used to calculate the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points;

[0073] A correction module 204 is used to perform posture correction on the face in the source image to align the face in the standard ID photo through matrix transformation and scale adjustment based on the translation deviation and the angle deviation;

[0074] The generation module 205 is used to crop the source image after face alignment to generate a target ID photo.

[0075] From the above attached Figure 2 It can be seen from the example of the ID photo generation device based on facial feature points that facial feature points are extracted from the face area of ​​the source image, and after the coordinates of the facial feature points are obtained, the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo are calculated based on the coordinates of the facial feature points. Based on these translation deviations and angle deviations, the posture of the face in the source image is corrected through matrix transformation and scale adjustment to align the face in the standard ID photo. Finally, the source image after face alignment is cropped to generate the target ID photo. Compared with the prior art method of manually adjusting the image by manually positioning the head and adjusting the posture, the technical solution of the present application can be automatically completed through an artificial intelligence model without manual participation or intervention, thereby improving the accuracy and efficiency of the posture alignment of the face in the source image with the face in the standard ID photo, especially for source images based on complex backgrounds or shot at angles, the alignment accuracy or effect is more obvious.

[0076] Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a method for generating a ID photo based on facial feature points. When the processor 30 executes the computer program 32, the steps in the embodiment of the method for generating an ID photo based on facial feature points are implemented, such as Figure 1Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 2 The functions of the detection module 201, the extraction module 202, the calculation module 203, the correction module 204 and the generation module 205 are shown.

[0077] Exemplarily, the computer program 32 of the ID photo generation method based on facial feature points mainly includes: detecting the face area of ​​the source image; extracting facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points; calculating the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points; based on the translation deviation and angle deviation, performing posture correction on the face in the source image through matrix transformation and scale adjustment to align the face in the standard ID photo; cropping the source image after face alignment to generate the target ID photo. The computer program 32 can be divided into one or more modules / units, one or more modules / units are stored in the memory 31 and executed by the processor 30 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 can be divided into the functions of a detection module 201, an extraction module 202, a calculation module 203, a correction module 204 and a generation module 205 (modules in the virtual device), and the specific functions of each module are as follows: a detection module 201, used to detect the face area of ​​the source image; an extraction module 202, used to extract facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points; a calculation module 203, used to calculate the translation deviation and angular deviation of the face in the source image relative to the face in the standard ID photo based on the coordinates of the facial feature points; a correction module 204, used to perform posture correction on the face in the source image through matrix transformation and scale adjustment based on the translation deviation and angular deviation to align the face in the standard ID photo; a generation module 205, used to crop the source image after face alignment to generate a target ID photo.

[0078] The electronic device 3 may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of the electronic device 3 and does not constitute a limitation of the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0079] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0080] The memory 31 may be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 3. Further, the memory 31 may also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0081] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0082] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0083] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0084] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program of the ID photo generation method based on facial feature points can be stored in a storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments, that is, detecting the face area of ​​the source image; extracting facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points; calculating the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points; based on the translation deviation and angle deviation, correcting the posture of the face in the source image through matrix transformation and scale adjustment to align the face in the standard ID photo; cropping the source image after face alignment to generate the target ID photo. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Storage media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media do not include electric carrier signals and telecommunication signals.

[0088] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application is described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present invention.

Claims

1. A method for generating ID photos based on facial feature points, characterized in that: The method comprises: Detect the face area of ​​the source image; Extracting facial feature points from the face area of ​​the source image to obtain coordinates of the facial feature points; Calculating the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points; Based on the translation deviation and the angle deviation, performing posture correction on the face in the source image through matrix transformation and scale adjustment to align the face in the standard ID photo; Crop the source image after face alignment to generate the target ID photo.

2. The ID photo generation method based on facial feature points as claimed in claim 1, characterized in that: The step of extracting facial feature points from the facial region of the source image to obtain coordinates of the facial feature points includes: Calculate the gradient values ​​I of the source image in the X-axis direction and the Y-axis direction of the two-dimensional coordinate system respectively x and I y ; Use Gaussian function and I x *I y Perform filtering to obtain matrices A, B and C respectively; According to matrices A, B and C, calculate the feature point response function R(x,y)=Det(M)-k*(trace(M)) 2 The value of R, Said Det(M) is the determinant of M, trace(M)=A+B is the trace of M, and k is an empirical parameter; The value R is subjected to non-maximum suppression, and facial feature points in the face area are extracted and their coordinates are recorded.

3. The ID photo generation method based on facial feature points as claimed in claim 1, characterized in that: The step of calculating the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points includes: According to the coordinates of the facial feature points, a horizontal distance difference and a vertical distance difference between the center of the face in the source image and the center of the face in the standard ID photo are calculated as a translation deviation of the face in the target image relative to the face in the standard ID photo; According to the coordinates of the facial feature points, the deviation angles of rotation around the z-axis, the deviation angles of rotation around the x-axis, and the deviation angles of rotation around the y-axis are calculated respectively as the angular deviations of the face in the source image relative to the face in the standard ID photo.

4. The ID photo generation method based on facial feature points as claimed in claim 1, characterized in that: The angle deviation includes a deviation angle of rotation about the z-axis, a deviation angle of rotation about the x-axis, and a deviation angle of rotation about the y-axis between the face in the source image and the face in the standard ID photo. The posture correction of the face in the source image through matrix transformation and scale adjustment based on the translation deviation and the angle deviation to align the face in the standard ID photo includes: According to the deviation angle of rotation around the z-axis, the deviation angle of rotation around the x-axis and the deviation angle of rotation around the y-axis of the face in the source image and the face in the standard ID photo, the corresponding first rotation matrix R is calculated respectively. yaw , the second rotation matrix R pitch , the third rotation matrix R roll And the final rotation matrix R is as follows: R=R yaw ×R pitch ×R roll The θ yaw is the deviation angle between the face in the source image and the face in the standard ID photo around the z-axis, and the θ pitch is the deviation angle between the face in the source image and the face in the standard ID photo around the x-axis, and the θ roll is the deviation angle between the face in the source image and the face in the standard ID photo rotated around the y-axis; The width W of the face in the standard ID photo std The width W of the face in the source image img , calculate the scale adjustment coefficient According to the formula Transform the source image until all deviations are corrected to below a preset threshold, wherein x and y are the horizontal and vertical coordinates of any pixel in the source image, respectively, and the affine transformation matrix The R 2D is the two-dimensional rotation matrix part extracted from the final rotation matrix R, the t x and t y are respectively the horizontal distance difference and the vertical distance difference between the center of the face in the source image and the center of the face in the standard ID photo.

5. The method for generating a ID photo based on facial feature points as claimed in claim 4, characterized in that: According to the formula Transforming the source image until all deviations are corrected to be below a preset threshold, comprising: Optimize the affine transformation matrix T based on the gradient descent algorithm affine-trans , until all deviations are corrected to below the preset threshold.

6. The ID photo generation method based on facial feature points as claimed in claim 5, characterized in that: The affine transformation matrix T is optimized based on the gradient descent algorithm. affine-trans , until all deviations are corrected to below a preset threshold, including: Defining an objective function, the objective function being used to measure a reprojection error between facial feature points in the source image and facial feature points in the standard ID photo; Calculate the objective function with respect to the affine transformation matrix T affine-trans The gradient of the parameters of ; According to the affine transformation matrix T affine-trans The gradient of the parameters is controlled by the learning rate to update the affine transformation matrix T affine-trans until the preset convergence condition is met.

7. The method for generating a ID photo based on facial feature points as claimed in claim 6, characterized in that: According to the affine transformation matrix T affine-trans The gradient of the parameters is controlled by the learning rate to update the affine transformation matrix T affine-trans Parameters until the preset convergence conditions are met include: Calculate first-order momentum and second-order momentum; According to the objective function relative to the affine transformation matrix T affine-trans The current gradient of the parameters of , adjusts the learning rate by modifying the first-order momentum and the second-order momentum.

8. A device for generating ID photos based on facial feature points, characterized in that: The device comprises: A detection module, used to detect the face area of ​​the source image; An extraction module, used to extract facial feature points from the face area of ​​the source image to obtain the coordinates of the facial feature points; A calculation module, used to calculate the translation deviation and angle deviation of the face in the source image relative to the face in the standard ID photo according to the coordinates of the facial feature points; A correction module, configured to perform posture correction on the face in the source image through matrix transformation and scale adjustment based on the translation deviation and the angle deviation so as to align the face in the standard ID photo; The generation module is used to crop the source image after face alignment to generate a target ID photo.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images