A method and device for verifying a category of an ID photo, an electronic device, and a storage medium

Abstract

The application provides a method and device for verifying a category of an ID photo, an electronic device and a storage medium, wherein the method comprises: obtaining a to-be-processed picture; performing face recognition on the to-be-processed picture to obtain a number of faces; when the number of faces is a first threshold, performing classification processing on the to-be-processed picture to obtain a first confidence; performing background segmentation on the to-be-processed picture to obtain a first sub-confidence; performing face feature point extraction on the to-be-processed picture to obtain a second sub-confidence; performing text recognition on the to-be-processed picture to obtain a third sub-confidence; obtaining a second confidence according to the first sub-confidence, the second sub-confidence and the third sub-confidence; and determining whether the to-be-processed picture is an ID photo according to the first confidence and the second confidence.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a method, apparatus, electronic device, and storage medium for verifying ID photo categories. Background Technology

[0002] Currently, before applying for a loan, users upload various information, including photos. To verify whether these photos are authentic ID photos, they need to be examined from multiple angles. Traditional verification methods use neural network models, confirming the authenticity of the photo based on the model's output confidence level. However, neural network models can only accurately verify the authenticity of uploaded photos in specific scenarios; for example, they can only precisely verify high-quality uploaded images.

[0003] Therefore, relying solely on neural networks is insufficient to accurately verify whether an image is an ID photo across all scenarios, thus increasing loan risk. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, apparatus, electronic device, and storage medium for verifying the type of ID photo. By employing multiple methods to jointly verify whether an image is an ID photo, the accuracy of the verification is improved. This method can accurately verify whether images uploaded by users in various scenarios are ID photos, thereby reducing loan risks.

[0005] To achieve the above objectives, a first aspect of this application provides a method for verifying the type of ID photo, the method comprising:

[0006] Get the image to be processed;

[0007] Perform facial recognition processing on the image to be processed to obtain the number of faces;

[0008] When the number of faces reaches the first threshold, the image to be processed is classified to obtain the first confidence score.

[0009] Perform background segmentation on the image to be processed to obtain the first sub-confidence score;

[0010] Facial feature points are extracted from the image to be processed to obtain the second sub-confidence score;

[0011] The image to be processed is subjected to text recognition processing to obtain the third sub-confidence score;

[0012] The second confidence level is obtained based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level;

[0013] Based on the first and second confidence levels, determine whether the image to be processed is an ID photo.

[0014] In conjunction with the first aspect, in one possible implementation, the image to be processed is classified to obtain a first confidence level, and the method further includes:

[0015] Feature extraction is performed on the image to be processed to obtain initial features, which include k sub-features;

[0016] Attention processing is applied to the target feature map to obtain the weight coefficient of each sub-feature;

[0017] Based on the weight coefficient of each sub-feature, the k sub-features are weighted to obtain the target feature;

[0018] Based on the target characteristics, the first confidence level is obtained.

[0019] In conjunction with the first aspect, in one possible implementation, feature extraction is performed on the image to be processed to obtain initial features, including:

[0020] Obtain the pixel features of the image to be processed in the RGB color space to obtain x sub-features;

[0021] Obtain the color features of the image to be processed in the HSV color space to obtain y sub-features;

[0022] Perform wavelet transform on the image to be processed to obtain z sub-features;

[0023] The initial features are obtained by concatenating x, y, and z sub-features, where k = x + y + z.

[0024] In conjunction with the first aspect, in one possible implementation, the classification of the image to be processed is achieved through an ID photo classification model, which is trained through the following steps:

[0025] Acquire training images;

[0026] Obtain multiple spectrograms of the training images;

[0027] The training images are input into the initial model to obtain multiple feature maps, including multiple spectrograms and multiple feature maps. Figure 1 There is a one-to-one correspondence, and the size of any one spectrogram is the same as the size of the first feature map corresponding to that spectrogram;

[0028] Calculate the mean square error of multiple spectrograms and multiple feature maps to obtain multiple mean square errors;

[0029] The average of multiple mean squared errors is used as the first loss;

[0030] The cross-entropy loss of multiple feature maps is determined based on multiple feature maps, and the cross-entropy loss is used as the second loss.

[0031] Based on the first loss and the second loss, the target loss is obtained;

[0032] Based on the target loss, the initial model is trained to obtain the ID photo classification model.

[0033] In conjunction with the first aspect, in one possible implementation, background segmentation is performed on the image to be processed to obtain a first sub-confidence score, including:

[0034] Perform background segmentation on the image to be processed to obtain sub-background images;

[0035] Perform grayscale processing on the sub-background image to obtain the corresponding grayscale value image of the sub-background image;

[0036] Calculate the average value of pixels in a grayscale image;

[0037] Calculate the number of pixels in the grayscale image that are greater than or equal to a first threshold and less than or equal to a second threshold, where the first threshold is less than the average value and the second threshold is greater than the average value;

[0038] The ratio of the number of pixels in the grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale image is calculated, and this ratio is used as the first sub-confidence.

[0039] In conjunction with the first aspect, in one possible implementation, facial feature points are extracted from the image to be processed to obtain a second sub-confidence score.

[0040] Facial feature points are extracted from the image to be processed, resulting in multiple facial feature points;

[0041] Select multiple target facial feature points from multiple facial feature points;

[0042] Obtain multiple two-dimensional pixel coordinates of multiple target facial feature points, wherein the multiple target facial feature points and the multiple two-dimensional pixel coordinates correspond one-to-one;

[0043] Calculate the rotation vectors of multiple target two-dimensional pixel coordinates and multiple three-dimensional spatial coordinates that correspond one-to-one with multiple facial feature points in the ID photo face;

[0044] Determine the rotation matrix based on the rotation vector;

[0045] The Eulerian distance is determined based on the rotation matrix, where the Eulerian distance is used to characterize the rotation angle of the face in the image to be processed.

[0046] The rotation angle of the face is normalized to obtain the second sub-confidence.

[0047] In conjunction with the first aspect, in one possible implementation, text recognition processing is performed on the image to be processed to obtain a third sub-confidence score, including:

[0048] The image to be processed is subjected to text recognition processing to obtain the first text;

[0049] Segment the first text to obtain at least one word;

[0050] Compare each word with the words in the regular expression template to obtain the number of words in the at least one word that match the regular expression template;

[0051] Calculate the ratio of the number of characters in at least one word that matches the regular expression template to the total number of characters in the regular expression template, and determine the ratio as the third sub-confidence.

[0052] In conjunction with the first aspect, in one possible implementation, obtaining the second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level includes:

[0053] The first, second, and third sub-confidence scores are weighted with coefficients to obtain the second confidence score.

[0054] In conjunction with the first aspect, in one possible implementation, determining whether the image to be processed is an ID photo based on a first confidence level and a second confidence level includes:

[0055] When both the first and second confidence levels fall within the target confidence level range, the image to be processed is determined to be an ID photo.

[0056] Otherwise, determine that the image to be processed is not an ID photo.

[0057] A second aspect of this application provides a method for verifying ID photo categories, the method being applied to an ID photo category verification device, the method comprising:

[0058] The acquisition unit is used to acquire the image to be processed.

[0059] The processing unit is used to perform face recognition processing on the image to be processed to obtain the number of faces;

[0060] The processing unit is also used to input the image to be processed into the ID photo classification model to obtain the first confidence score when the number of faces is the first threshold.

[0061] The processing unit is also used to perform background segmentation on the image to be processed to obtain the first sub-confidence score;

[0062] The processing unit is also used to extract facial feature points from the image to be processed to obtain a second sub-confidence score;

[0063] The processing unit is also used to perform text recognition processing on the image to be processed to obtain a third sub-confidence score;

[0064] The processing unit is also configured to obtain a second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level;

[0065] The processing unit is also used to determine whether the image to be processed is an ID photo based on a first confidence level and a second confidence level.

[0066] A third aspect of this application provides an electronic device including an input device and an output device, and further including a processing chip adapted to implement one or more instructions; and a memory storing one or more computer programs adapted to be loaded by the processing chip and executed as in the method of the first aspect described above.

[0067] A fourth aspect of this application provides a computer storage medium storing one or more instructions adapted for loading by a processing chip and executing the steps of the method described in the first aspect above.

[0068] The above-mentioned solution in this application includes at least the following beneficial effects:

[0069] In this embodiment, multiple verification methods are used to ensure that the image uploaded by the user is an ID photo. First, the uploaded image is input into an ID photo classification model to obtain a first confidence score. Then, background segmentation is performed on the image to determine a first sub-confidence score. Next, facial feature point extraction is performed on the uploaded image to calculate a second sub-confidence score. Then, text recognition processing is performed to calculate a third sub-confidence score. Finally, a second confidence score is obtained based on the first, second, and third sub-confidence scores. When both the first and second confidence scores fall within the target confidence score range, the image is determined to be an ID photo. This multi-method verification improves the accuracy of the results, thereby reducing loan risk. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 A system architecture diagram of a method for verifying ID photo categories provided in this application embodiment;

[0072] Figure 2A specific application scenario diagram of a method for verifying the type of ID photo provided in this application embodiment;

[0073] Figure 3 A flowchart illustrating a method for verifying the type of ID photo provided in an embodiment of this application;

[0074] Figure 4 An initial feature diagram provided for an embodiment of this application;

[0075] Figure 5 A schematic diagram of a sub-background image provided for an embodiment of this application;

[0076] Figure 6 A schematic diagram of a grayscale image of a sub-background image provided in an embodiment of this application;

[0077] Figure 7 A schematic diagram of a target facial feature map provided in an embodiment of this application;

[0078] Figure 8 A schematic diagram illustrating a complete process provided for an embodiment of this application;

[0079] Figure 9 A schematic diagram of the structure of an ID photo category verification device provided in this application embodiment;

[0080] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0081] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0082] The terms "comprising" and "having," and any variations thereof, appearing in this specification, claims, and drawings, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. Furthermore, the terms "first," "second," and "third," etc., are used to distinguish different objects and are not used to describe a specific order.

[0083] Please see Figure 1 , Figure 1 This application provides a system architecture diagram for a method of verifying ID photo categories. The system architecture diagram includes a user terminal 10, a cloud server 20, and an ID photo category verification device 30.

[0084] The user terminal 10 can be an input device such as a smartphone, tablet, or desktop computer. Users fill in the required information and upload images on the user terminal 10.

[0085] The cloud server 20 receives various materials and images uploaded by the user terminal 10, sends the images to the ID photo verification device 30 for verification, and receives the verification results. Finally, it sends the verification results back to the user terminal 10.

[0086] The ID photo type verification device 30 receives an image from the user of the client 10 sent by the cloud server 20 and verifies whether the image is an ID photo. If it is an ID photo, it determines the type of ID photo and sends the result back to the cloud server 20.

[0087] Please see Figure 2 , Figure 2 This diagram illustrates a specific application scenario for a method of verifying document photo categories provided in this application embodiment. The scenario involves a user, specifically a loan officer, inputting the required loan information and images onto a desktop computer and receiving the results.

[0088] The specific scenarios are as follows:

[0089] First, users log in to the loan mini-programs or apps of major banks or loan companies on their desktop computers. They then select the "Loan" option in the mini-program or app, enter the required information, and upload images. After completing the information and image input, they wait for the mini-program or app to provide verification results, indicating whether the images meet the requirements and are valid identification photos. If the result confirms the image is an identification photo, the loan application process proceeds. If the result indicates the image is not an identification photo, the user needs to re-upload the image.

[0090] Finally, the ID photo category verification method proposed in this application can also be applied to other ID photo verification scenarios, such as user authentication and initial registration in confidential locations. Furthermore, the user can change depending on the scenario. For example, for user authentication, the ID photo category verification method proposed in this application can be changed to a student ID photo category verification method; similarly, for confidential locations, the ID photo category verification method proposed in this application can be changed to a confidential location ID photo category verification method. In this embodiment, a confidential workshop will be used as an example. Taking a user applying for a loan as an example, the ID photo category verification method proposed in this application will be explained. The management methods in other scenarios are similar to the ID photo category verification method in confidential location scenarios, and will not be elaborated further here.

[0091] Please see Figure 3 , Figure 3 An interactive diagram illustrating a method for verifying ID photo categories provided in this application embodiment. This method is applied to an ID photo category verification device, such as... Figure 3 As shown, steps 301-308 are included:

[0092] 301: Get the image to be processed.

[0093] In this embodiment, the ID photo category verification device obtains an ID photo sent by a cloud server, which is an ID photo uploaded by the user on the user's terminal.

[0094] 302: Perform face recognition processing on the image to be processed to obtain the number of faces.

[0095] In this embodiment, the image to be processed is first subjected to facial recognition processing. Based on the number of faces identified, non-ID photos are filtered out. If the number of faces is greater than one, the image to be processed is not an ID photo. ID photos typically contain only one face image; if the number of faces is greater than one, it cannot be an ID photo. Therefore, facial recognition processing is performed first, and some non-ID photos are excluded based on the number of faces. This allows for rapid filtering of images that do not meet the criteria.

[0096] 303: When the number of faces is the first threshold, the image to be processed is classified to obtain the first confidence score.

[0097] In this embodiment, given that the number of faces is a first threshold, the image to be processed is first subjected to feature extraction to obtain initial features. These initial features include k sub-features.

[0098] The initial features are obtained as follows:

[0099] First, obtain the pixel features of the image to be processed in the RGB color space to obtain x sub-features.

[0100] Then, the color features of the image to be processed in the HSV color space are obtained, resulting in y sub-features.

[0101] Then perform wavelet transform on the image to be processed to obtain z sub-features.

[0102] Finally, the x, y, and z sub-features are concatenated to obtain the initial features, where k = x + y + z. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of an initial feature provided for an embodiment of this application.

[0103] Then, attention processing is performed on the target feature map to obtain the weight coefficient of each sub-feature.

[0104] Then, based on the weight coefficient of each sub-feature, the k sub-features are weighted to obtain the target feature. Finally, based on the target feature, the first confidence score is obtained.

[0105] In this embodiment, when the number of faces is 1, the image to be processed is classified. The first threshold is 1. The classification of the image to be processed utilizes an ID photo classification model, the training process of which is as follows:

[0106] First, training images are acquired. Then, multiple spectrograms of the training images are acquired; these spectrograms are obtained by performing a Fourier transform on the training images. Next, the training images are input into an initial model to obtain multiple feature maps. The initial model performs a convolution operation on the training images to obtain multiple feature maps. These multiple spectrograms and multiple feature maps... Figure 1 There is a one-to-one correspondence, and the size of any spectrogram is the same as the size of the first feature map corresponding to that spectrogram. The mean squared errors of multiple spectrograms and multiple feature maps are calculated, resulting in multiple mean squared errors. The average of these multiple mean squared errors is used as the first loss. The cross-entropy loss of the multiple feature maps is determined based on the multiple feature maps, and this cross-entropy loss is used as the second loss. The target loss is obtained based on the first and second losses. The initial model is trained using the target loss to obtain the ID photo classification model.

[0107] After processing by the ID photo classification model, the first confidence score is obtained, which is a probability value that the image to be processed is an ID photo.

[0108] 304: Perform background segmentation on the image to be processed to obtain the first sub-confidence score.

[0109] In this embodiment, firstly, the image to be processed is segmented to obtain sub-background images, such as... Figure 5As shown, Figure 5 This is a schematic diagram of a sub-background image provided in an embodiment of this application. Then, the sub-background image is subjected to grayscale processing to obtain a grayscale value image corresponding to the sub-background image, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of a grayscale image of a sub-background image provided in an embodiment of this application. The average value of the pixels in the grayscale image is then calculated. The number of pixels in the grayscale image that are greater than or equal to a first threshold and less than or equal to a second threshold is calculated, where the first threshold is less than the average value and the second threshold is greater than the average value, and the first and second thresholds differ by 10 units to the left and right of the average value. The ratio of the number of pixels in the grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale image is calculated, and this ratio is used as the first sub-confidence level.

[0110] Typically, ID photos have a relatively simple background, either a single color or the same background image as the ID card. The background of the image being processed is determined by calculating the average grayscale value of the background area and the ratio of the number of grayscale values ​​that differ from the average by 10 to the total number of grayscale values. If a large number of grayscale values ​​are near the average, it means the background is simple, which implies it's an ID photo. The probability of the image being an ID photo can be determined through background image processing.

[0111] 305: Extract facial feature points from the image to be processed to obtain the second sub-confidence score.

[0112] In this embodiment, firstly, facial feature points are extracted from the image to be processed, resulting in multiple facial feature points. Then, multiple target facial feature points are selected from these multiple facial feature points, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of a target facial feature map provided in an embodiment of this application. The target facial feature points include six specific points: the left corner of the eye, the right corner of the eye, the left corner of the mouth, the right corner of the mouth, the tip of the nose, and the chin. Next, multiple two-dimensional pixel coordinates of the multiple target facial feature points are obtained, with each target facial feature point corresponding to one of the multiple two-dimensional pixel coordinates. Then, rotation vectors are calculated for the multiple two-dimensional pixel coordinates of the targets and the multiple three-dimensional spatial coordinates corresponding to the multiple facial feature points in the ID photo. Based on the rotation vectors, a rotation matrix is ​​determined. Based on the rotation matrix, the Eulerian distance is determined, whereby the Eulerian distance characterizes the rotation angle of the face in the image to be processed. Finally, the rotation angle of the face is normalized to obtain a second sub-confidence score.

[0113] For example, firstly, 68 key points on the head are detected, from which 6 target feature points are selected: the left corner of the eye, the right corner of the eye, the left corner of the mouth, the right corner of the mouth, the tip of the nose, and the chin. The 2D pixel coordinates and their corresponding 3D spatial coordinates for each target feature point are obtained. Specifically, the real 3D spatial coordinates of the 6 target feature points are: tip of the nose (0.0, 0.0, 0.0), chin (0.0, -330.0, -65.0), left corner of the eye (-225.0, 170.0, -135.0), right corner of the eye (225.0, 170.0, -135.0), left corner of the mouth (-150.0, ...).

[0114] -150.0, -125.0), right corner of the mouth (150.0, -150.0, -125.0).

[0115] Based on the two-dimensional pixel coordinates and three-dimensional spatial coordinates of six target feature points, a rotation vector is calculated. A rotation matrix is ​​then determined based on the rotation vector. The Eulerian distance is calculated using the rotation matrix to represent the rotation angle of the face in the image to be processed. Finally, the rotation angle of the face is normalized. The normalization formula is as follows:

[0116]

[0117] Where r2 is the second sub-confidence, and a, b, and c are the perspectives of pitch, yaw, and roll.

[0118] Since the extreme angles of face rotation are: Pitch: -60.4~69.6, Yaw: -79.8~75.3, Roll: -40.9~63.3, the normalized weight allocation is 70, 80, 65.

[0119] 306: Perform text recognition processing on the image to be processed to obtain the third sub-confidence score.

[0120] In this embodiment, firstly, text recognition processing is performed on the image to be processed to obtain the first text.

[0121] Then, the first text is segmented to obtain at least one word.

[0122] Finally, each word is compared with the words in the regular expression template to obtain the number of words in the at least one word that match the regular expression template. The regular expression template includes "name", "gender", "address", "birth date", "number", and an 18-digit numeric string.

[0123] Calculate the ratio of the number of characters in at least one word that matches the regular expression template to the total number of characters in the regular expression template, and determine the ratio as the third sub-confidence.

[0124] 307: The second confidence level is obtained based on the first, second, and third sub-confidence levels.

[0125] The first, second, and third sub-confidence scores are weighted by coefficients to obtain the second confidence score. The weight of the second sub-confidence score is equal to the weight of the first sub-confidence score, and the weight of the first sub-confidence score is greater than the weight of the third sub-confidence score. In this embodiment, ID photos may also lack text; therefore, the weight of the third sub-confidence score is relatively smaller than that of the first and second sub-confidence scores. Typically, a typical ID photo has a simple background, a straight face, and a small rotation angle. Therefore, the third sub-confidence score has a smaller weight when assigning weights.

[0126] 308: Determine whether the image to be processed is an ID photo based on the first and second confidence levels.

[0127] In this embodiment, both the first confidence level and the second confidence level are probability values ​​for whether the image to be processed is an ID photo. To ensure the reasonableness of the result, the image to be processed is only determined to be an ID photo when both the first confidence level and the second confidence level fall within the target confidence level range. Otherwise, the image to be processed is determined not to be an ID photo.

[0128] In another embodiment, when the third sub-confidence score is greater than the third threshold, the image to be processed is determined to be an ID card photo. An ID card photo typically contains a single background, a small face rotation angle, and text from a regular expression template. After background segmentation and face rotation angle calculation, the image to be processed can be determined to be an ID card photo. However, whether it is an ID card photo requires text processing. To ensure that the image to be processed is an ID card photo, it is also necessary to ensure that the ratio of the number of characters in at least one word in the image that matches the regular expression template to the total number of characters in the regular expression template is greater than the third threshold.

[0129] In another embodiment, such as Figure 8 As shown, Figure 8This is a schematic diagram illustrating a complete process provided in an embodiment of this application. First, face recognition is performed on the image to be processed. If the number of faces is 1, subsequent operations are performed. If the number of faces is not 1, the image to be processed is not an ID photo. Then, the image to be processed undergoes two aspects of processing: First, the user-uploaded image is input into an ID photo classification model to obtain a first confidence level. Then, background segmentation is performed on the image to determine whether the grayscale values ​​in the background image are concentrated at a single value. Because ID photo backgrounds are generally solid colors, the grayscale values ​​should be concentrated at a certain value. The ratio of the number of pixels concentrated near the average range to the total number of pixels is used as the first sub-confidence level. Next, facial feature points are extracted from the user-uploaded image. The two-dimensional pixel coordinates of the target feature point are selected from 68 facial feature points, and the Eulerian distance of the face is calculated using the three-dimensional spatial coordinates of the target feature point. The Eulerian distance is used to characterize the rotation angle of the face. The normalized value of the rotation angle is used as the second sub-confidence level. Next, text recognition processing is performed, and the ratio of characters overlapping with the regular expression template is used as the third sub-confidence. Finally, the second confidence is calculated based on the first, second, and third sub-confidences. When both the first and second confidences fall within the target confidence interval, the image is determined to be an ID photo.

[0130] As can be seen, this embodiment of the application facilitates the use of multiple verification methods to ensure that the image uploaded by the user is an ID photo. First, the user-uploaded image is input into an ID photo classification model to obtain a first confidence level. Then, background segmentation is performed on the image to determine a first sub-confidence level. Next, facial feature point extraction is performed on the user-uploaded image to calculate a second sub-confidence level. Then, text recognition processing is performed to calculate a third sub-confidence level. Finally, a second confidence level is obtained based on the first, second, and third sub-confidence levels. When both the first and second confidence levels fall within the target confidence level range, the image is determined to be an ID photo. This multi-method verification improves the accuracy of the results, thereby reducing loan risk.

[0131] Please see Figure 9 , Figure 9 A schematic diagram of the structure of an ID photo category verification device provided in this application embodiment is shown below. Figure 9 As shown, the ID photo verification device includes an acquisition unit 901 and a processing unit 902; wherein:

[0132] The acquisition unit is used to acquire the image to be processed.

[0133] The processing unit is used to perform face recognition processing on the image to be processed to obtain the number of faces;

[0134] The processing unit is also used to input the image to be processed into the ID photo classification model to obtain the first confidence score when the number of faces is the first threshold.

[0135] The processing unit is also used to perform background segmentation on the image to be processed to obtain the first sub-confidence score;

[0136] The processing unit is also used to extract facial feature points from the image to be processed to obtain a second sub-confidence score;

[0137] The processing unit is also used to perform text recognition processing on the image to be processed to obtain a third sub-confidence score;

[0138] The processing unit is also configured to obtain a second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level;

[0139] The processing unit is also used to determine whether the image to be processed is an ID photo based on a first confidence level and a second confidence level.

[0140] It can be seen that,

[0141] In one possible implementation, in classifying the image to be processed to obtain a first confidence level, the processing unit 902 is specifically used for:

[0142] Feature extraction is performed on the image to be processed to obtain initial features, which include k sub-features;

[0143] Attention processing is applied to the target feature map to obtain the weight coefficient of each sub-feature;

[0144] Based on the weight coefficient of each sub-feature, the k sub-features are weighted to obtain the target feature;

[0145] Based on the target characteristics, the first confidence level is obtained.

[0146] In one possible implementation, in extracting features from the image to be processed to obtain initial features, the processing unit 902 is specifically used for:

[0147] Obtain the pixel features of the image to be processed in the RGB color space to obtain x sub-features;

[0148] Obtain the color features of the image to be processed in the HSV color space to obtain y sub-features;

[0149] Perform wavelet transform on the image to be processed to obtain z sub-features;

[0150] The initial features are obtained by concatenating x, y, and z sub-features, where k = x + y + z.

[0151] In one possible implementation, the classification of the image to be processed is achieved through an ID photo classification model, which is trained through the following steps. Specifically, the processing unit 902 is used for:

[0152] Acquire training images;

[0153] Obtain multiple spectrograms of the training images;

[0154] The training images are input into the initial model to obtain multiple feature maps, including multiple spectrograms and multiple feature maps. Figure 1 There is a one-to-one correspondence, and the size of any one spectrogram is the same as the size of the first feature map corresponding to that spectrogram;

[0155] Calculate the mean square error of multiple spectrograms and multiple feature maps to obtain multiple mean square errors;

[0156] The average of multiple mean squared errors is used as the first loss;

[0157] The cross-entropy loss of multiple feature maps is determined based on multiple feature maps, and the cross-entropy loss is used as the second loss.

[0158] Based on the first loss and the second loss, the target loss is obtained;

[0159] Based on the target loss, the initial model is trained to obtain the ID photo classification model.

[0160] In one possible implementation, in performing background segmentation on the image to be processed to obtain a first sub-confidence score, the processing unit 902 is specifically used for:

[0161] Perform background segmentation on the image to be processed to obtain sub-background images;

[0162] Perform grayscale processing on the sub-background image to obtain the corresponding grayscale value image of the sub-background image;

[0163] Calculate the average value of pixels in a grayscale image;

[0164] Calculate the number of pixels in the grayscale image that are greater than or equal to a first threshold and less than or equal to a second threshold, where the first threshold is less than the average value and the second threshold is greater than the average value;

[0165] The ratio of the number of pixels in the grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale image is used as the first sub-confidence score.

[0166] In one possible implementation, in extracting facial feature points from the image to be processed and obtaining a second sub-confidence score, the processing unit 902 is specifically used for:

[0167] Facial feature points are extracted from the image to be processed, resulting in multiple facial feature points;

[0168] Select multiple target facial feature points from multiple facial feature points;

[0169] Obtain multiple two-dimensional pixel coordinates of multiple target facial feature points, wherein the multiple target facial feature points and the multiple two-dimensional pixel coordinates correspond one-to-one;

[0170] Calculate the rotation vectors of multiple target two-dimensional pixel coordinates and multiple three-dimensional spatial coordinates that correspond one-to-one with multiple facial feature points in the ID photo face;

[0171] Determine the rotation matrix based on the rotation vector;

[0172] The Eulerian distance is determined based on the rotation matrix, where the Eulerian distance is used to characterize the rotation angle of the face in the image to be processed.

[0173] The rotation angle of the face is normalized to obtain the second sub-confidence.

[0174] In one possible implementation, in performing text recognition processing on the image to be processed to obtain a third sub-confidence score, the processing unit 902 is specifically used for:

[0175] The image to be processed is subjected to text recognition processing to obtain the first text;

[0176] Segment the first text to obtain at least one word;

[0177] Compare each word with the words in the regular expression template to obtain the number of words in the at least one word that match the regular expression template;

[0178] Calculate the ratio of the number of characters in at least one word that matches the regular expression template to the total number of characters in the regular expression template, and determine the ratio as the third sub-confidence.

[0179] In one possible implementation, in obtaining the second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level, the processing unit 902 is specifically configured to:

[0180] The first, second, and third sub-confidence scores are weighted with coefficients to obtain the second confidence score.

[0181] In one possible implementation, in determining whether the image to be processed is an ID photo based on a first confidence level and a second confidence level, the processing unit 902 is specifically used for:

[0182] When both the first and second confidence levels fall within the target confidence level range, the image to be processed is determined to be an ID photo.

[0183] Otherwise, determine that the image to be processed is not an ID photo.

[0184] According to one embodiment of this application, Figure 9 The cloud server shown can be constructed by combining its various units into one or more other units, or one or more of these units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the cloud server may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0185] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 9 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 10 The document describes a photo identification verification device and a method for implementing photo identification verification according to embodiments of this application. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the same medium, and executed therein.

[0186] Based on the description of the method and apparatus embodiments above, this application also provides an electronic device. Please refer to... Figure 10 The electronic device includes at least a processor 1001, an input device 1002, an output device 1003, and a memory 1004. The processor 1001, input device 1002, output device 1003, and memory 1004 within the electronic device can be connected via a bus or other means.

[0187] The memory 1004 can be stored in the memory of the electronic device. The memory 1004 is used to store computer programs, which include program instructions. The processor 1001 is used to execute the program instructions stored in the memory 1004. The processor 1001 (or CPU (Central Processing Unit)) is the computing and control core of the electronic device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to achieve the corresponding method flow or corresponding function.

[0188] In one embodiment, the processor 1001 of the electronic device provided in this application can be used to perform a series of methods for verifying ID photo categories:

[0189] It can be seen that, Figure 10 Among the electronic devices shown, the smart screens in the workshop can display prompts in scenarios where the use of general electronic devices such as mobile phones is limited. This reminds staff to pick up packages as soon as possible, enabling automatic notification for packages in areas where communication tools are restricted. Simultaneously, it helps to reduce the clutter of various packages at package collection points, decreasing the risk of damage or misdelivery. Furthermore, installing call devices at package collection points allows recipients and couriers to contact each other quickly and resolve issues.

[0190] In one embodiment, processor 1001 executes:

[0191] Get the image to be processed;

[0192] Perform facial recognition processing on the image to be processed to obtain the number of faces;

[0193] When the number of faces reaches the first threshold, the image to be processed is classified to obtain the first confidence score.

[0194] Perform background segmentation on the image to be processed to obtain the first sub-confidence score;

[0195] Facial feature points are extracted from the image to be processed to obtain the second sub-confidence score;

[0196] The image to be processed is subjected to text recognition processing to obtain the third sub-confidence score;

[0197] The second confidence level is obtained based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level;

[0198] Based on the first and second confidence levels, determine whether the image to be processed is an ID photo.

[0199] In another embodiment, the image to be processed is classified to obtain a first confidence level, and the processor 1001 executes:

[0200] Feature extraction is performed on the image to be processed to obtain initial features, which include k sub-features;

[0201] Attention processing is applied to the target feature map to obtain the weight coefficient of each sub-feature;

[0202] Based on the weight coefficient of each sub-feature, the k sub-features are weighted to obtain the target feature;

[0203] Based on the target characteristics, the first confidence level is obtained.

[0204] In another embodiment, the processor 1001 performs feature extraction on the image to be processed to obtain initial features, including:

[0205] Obtain the pixel features of the image to be processed in the RGB color space to obtain x sub-features;

[0206] Obtain the color features of the image to be processed in the HSV color space to obtain y sub-features;

[0207] Perform wavelet transform on the image to be processed to obtain z sub-features;

[0208] The initial features are obtained by concatenating x, y, and z sub-features, where k = x + y + z.

[0209] In another embodiment, the classification processing of the image to be processed is achieved through an ID photo classification model. The processor 1001 executes the ID photo classification model to be trained through the following steps:

[0210] Acquire training images;

[0211] Obtain multiple spectrograms of the training images;

[0212] The training images are input into the initial model to obtain multiple feature maps, including multiple spectrograms and multiple feature maps. Figure 1 There is a one-to-one correspondence, and the size of any one spectrogram is the same as the size of the first feature map corresponding to that spectrogram;

[0213] Calculate the mean square error of multiple spectrograms and multiple feature maps to obtain multiple mean square errors;

[0214] The average of multiple mean squared errors is used as the first loss;

[0215] The cross-entropy loss of multiple feature maps is determined based on multiple feature maps, and the cross-entropy loss is used as the second loss.

[0216] Based on the first loss and the second loss, the target loss is obtained;

[0217] Based on the target loss, the initial model is trained to obtain the ID photo classification model.

[0218] In another embodiment, the processor 1001 performs background segmentation on the image to be processed to obtain a first sub-confidence score, including:

[0219] Perform background segmentation on the image to be processed to obtain sub-background images;

[0220] Perform grayscale processing on the sub-background image to obtain the corresponding grayscale value image of the sub-background image;

[0221] Calculate the average value of pixels in a grayscale image;

[0222] Calculate the number of pixels in the grayscale image that are greater than or equal to a first threshold and less than or equal to a second threshold, where the first threshold is less than the average value and the second threshold is greater than the average value;

[0223] The ratio of the number of pixels in the grayscale image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale image is used as the first sub-confidence score.

[0224] In another embodiment, the processor 1001 extracts facial feature points from the image to be processed according to the display sequence number to obtain a second sub-confidence score, including:

[0225] Facial feature points are extracted from the image to be processed, resulting in multiple facial feature points;

[0226] Select multiple target facial feature points from multiple facial feature points;

[0227] Obtain multiple two-dimensional pixel coordinates of multiple target facial feature points, wherein the multiple target facial feature points and the multiple two-dimensional pixel coordinates correspond one-to-one;

[0228] Calculate the rotation vectors of multiple target two-dimensional pixel coordinates and multiple three-dimensional spatial coordinates that correspond one-to-one with multiple facial feature points in the ID photo face;

[0229] Determine the rotation matrix based on the rotation vector;

[0230] The Eulerian distance is determined based on the rotation matrix, where the Eulerian distance is used to characterize the rotation angle of the face in the image to be processed.

[0231] The rotation angle of the face is normalized to obtain the second sub-confidence.

[0232] In another embodiment, the processor 1001 performs text recognition processing on the image to be processed to obtain a third sub-confidence score, including:

[0233] The image to be processed is subjected to text recognition processing to obtain the first text;

[0234] Segment the first text to obtain at least one word;

[0235] Compare each word with the words in the regular expression template to obtain the number of words in the at least one word that match the regular expression template;

[0236] Calculate the ratio of the number of characters in at least one word that matches the regular expression template to the total number of characters in the regular expression template, and determine the ratio as the third sub-confidence.

[0237] In another embodiment, the processor 1001 obtains a second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level, and executes:

[0238] When both the first and second confidence levels fall within the target confidence level range, the image to be processed is determined to be an ID photo.

[0239] Otherwise, determine that the image to be processed is not an ID photo.

[0240] For example, the electronic device may be an ultrasound examination device, a computer, etc., and includes, but is not limited to, a processor 1001, an input device 1002, an output device 1003, and a memory 1004. It may also include memory, a power supply, an application client module, etc. The input device 1002 may be a scanning device, a keyboard, a touchscreen, an RF receiver, etc., and the output device 1003 may be a speaker, a display, an RF transmitter, etc. Those skilled in the art will understand that the schematic diagram is merely an example of an electronic device and does not constitute a limitation on the electronic device; it may include more or fewer components than illustrated, or combine certain components, or use different components.

[0241] It should be noted that since the processor 1001 of the electronic device implements the steps in the above-described method when executing the computer program, the embodiments of the above-described method are all applicable to the electronic device and can achieve the same or similar beneficial effects.

[0242] This application embodiment also provides a computer storage medium (Memory), which is a memory device in an electronic device used to store programs and data. It is understood that the computer storage medium here can include the built-in storage medium in a terminal, or it can include an extended storage medium supported by the terminal. The computer storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by the processor 1001. These instructions can be one or more computer programs (including program code). It should be noted that the computer storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor 1001. In one embodiment, the processor 1001 can load and execute one or more instructions stored in the computer storage medium to implement the corresponding steps of the above-described method for verifying document photo categories.

[0243] For example, a computer program on a computer storage medium includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0244] It should be noted that since the computer program on the computer storage medium is executed by the processor to implement the steps in the above-described method for verifying the type of ID photo, all embodiments of the above-described method for verifying the type of ID photo are applicable to the computer storage medium and can achieve the same or similar beneficial effects.

[0245] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for verifying the type of ID photo, characterized in that, The method includes: Get the image to be processed; The image to be processed is subjected to face recognition processing to obtain the number of faces; When the number of faces reaches the first threshold, the image to be processed is classified to obtain the first confidence level. The process of performing background segmentation on the image to be processed to obtain a first sub-confidence score includes: performing background segmentation on the image to be processed to obtain a sub-background image; performing grayscale processing on the sub-background image to obtain a grayscale value image corresponding to the sub-background image; calculating the average value of the pixels in the grayscale value image; calculating the number of pixels in the grayscale value image that are greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is less than the average value and the second threshold is greater than the average value; and calculating the ratio of the number of pixels in the grayscale value image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale value image, and using the ratio as the first sub-confidence score. Facial feature points are extracted from the image to be processed to obtain the second sub-confidence score; The image to be processed is subjected to text recognition processing to obtain a third sub-confidence score; The second confidence level is obtained based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level; Determining whether the image to be processed is an ID photo based on the first confidence level and the second confidence level includes: determining that the image to be processed is an ID photo when both the first confidence level and the second confidence level fall within the target confidence level range; otherwise, determining that the image to be processed is not an ID photo.

2. The method according to claim 1, characterized in that, The process of classifying the image to be processed to obtain a first confidence level includes: Feature extraction is performed on the image to be processed to obtain initial features, wherein the initial features include k sub-features; Attention processing is performed on the image to be processed to obtain the weight coefficient of each sub-feature; The k sub-features are weighted according to the weight coefficient of each sub-feature to obtain the target feature; Based on the target features, a first confidence level is obtained.

3. The method according to claim 2, characterized in that, The step of extracting features from the image to be processed to obtain initial features includes: Obtain the pixel features of the image to be processed in the RGB color space to obtain x sub-features; Obtain the color features of the image to be processed in the HSV color space to obtain y sub-features; Perform wavelet transform on the image to be processed to obtain z sub-features; The x sub-features, y sub-features, and z sub-features are concatenated to obtain the initial features, where k = x + y + z.

4. The method according to any one of claims 1-3, characterized in that, The classification of the images to be processed is achieved through an ID photo classification model, which is trained through the following steps: Acquire training images; Obtain multiple spectrograms of the training images; The training image is input into the initial model to obtain multiple feature maps, wherein the multiple spectrograms and the multiple feature maps correspond one-to-one, and the size of any spectrogram is the same as the size of the first feature map corresponding to that spectrogram; Calculate the mean square error of the multiple spectrograms and the multiple feature maps to obtain multiple mean square errors; The average of the multiple mean square errors is used as the first loss; The cross-entropy loss of the multiple feature maps is determined based on the multiple feature maps, and the cross-entropy loss is used as the second loss; Based on the first loss and the second loss, the target loss is obtained; The initial model is trained based on the target loss to obtain the ID photo classification model.

5. The method according to claim 1, characterized in that, The step of extracting facial feature points from the image to be processed to obtain the second sub-confidence score includes: Facial feature points are extracted from the image to be processed to obtain multiple facial feature points; Select multiple target facial feature points from the plurality of facial feature points; Obtain multiple two-dimensional pixel coordinates of the multiple target facial feature points, wherein the multiple target facial feature points and the multiple two-dimensional pixel coordinates correspond one-to-one; Calculate the rotation vectors of the multiple two-dimensional pixel coordinates and the multiple three-dimensional spatial coordinates that correspond one-to-one with the multiple facial feature points in the ID photo face; Determine the rotation matrix based on the rotation vector; Based on the rotation matrix, the Eulerian distance is determined, wherein the Eulerian distance is used to characterize the rotation angle of the face in the image to be processed; The rotation angle of the face is normalized to obtain the second sub-confidence.

6. The method according to claim 1, characterized in that, The step of performing text recognition processing on the image to be processed to obtain a third sub-confidence score includes: The image to be processed is subjected to text recognition processing to obtain the first text; The first text is segmented to obtain at least one word; The at least one word is compared with the words in the regular expression template to obtain the number of words in the at least one word that match the regular expression template; Calculate the ratio of the number of characters in the at least one word that matches the regular expression template to the total number of characters in the regular expression template, and determine the ratio as the third sub-confidence.

7. The method according to claim 1, characterized in that, The step of obtaining the second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level includes: The first sub-confidence, the second sub-confidence, and the third sub-confidence are weighted with coefficients to obtain the second confidence.

8. The method according to claim 1, characterized in that, The step of determining whether the image to be processed is an ID photo based on the first confidence level and the second confidence level includes: When both the first confidence level and the second confidence level fall within the target confidence level range, the image to be processed is determined to be an ID photo. Otherwise, it is determined that the image to be processed is not an ID photo.

9. A device for verifying the type of ID photo, characterized in that, include: The acquisition unit is used to acquire the image to be processed. The processing unit is used to perform face recognition processing on the image to be processed to obtain the number of faces; The processing unit is also used to input the image to be processed into the ID photo classification model to obtain a first confidence level when the number of faces is a first threshold. The processing unit is further configured to perform background segmentation on the image to be processed to obtain a first sub-confidence score, including: performing background segmentation on the image to be processed to obtain a sub-background image; performing grayscale processing on the sub-background image to obtain a grayscale value image corresponding to the sub-background image; calculating the average value of the pixels in the grayscale value image; calculating the number of pixels in the grayscale value image that are greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is less than the average value and the second threshold is greater than the average value; calculating the ratio of the number of pixels in the grayscale value image that are greater than or equal to the first threshold and less than or equal to the second threshold to the total number of pixels in the grayscale value image, and using the ratio as the first sub-confidence score; The processing unit is also used to extract facial feature points from the image to be processed to obtain a second sub-confidence score; The processing unit is also used to perform text recognition processing on the image to be processed to obtain a third sub-confidence score; The processing unit is further configured to obtain a second confidence level based on the first sub-confidence level, the second sub-confidence level, and the third sub-confidence level; The processing unit is further configured to determine whether the image to be processed is an ID photo based on the first confidence level and the second confidence level, including: when both the first confidence level and the second confidence level fall within the target confidence level range, determining that the image to be processed is an ID photo; otherwise, determining that the image to be processed is not an ID photo.

10. An electronic device, characterized in that, include: A processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, and the processor being used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that is executed by a processor to implement the method as described in any one of claims 1-8.

Images