Card witness figure printing missing detection method and device

Abstract

The invention discloses a card certificate image printing missing detection method and device, relates to the technical field of card certificate printing missing detection, and aims at respectively carrying out portrait detection in an electronic printing original image and a to-be-detected image to determine the position of a portrait. And cutting to obtain an electronic printing portrait image and a to-be-detected portrait image, and performing feature point matching. And calculating an affine transformation matrix from the electronic printing portrait image to the portrait image to be detected, and performing affine transformation on the electronic printing portrait image to obtain an aligned portrait image. And respectively processing the portrait image to be detected and the aligned portrait image to generate respective binary matting mask images. And performing AND operation on the two binary cutout mask images to obtain a region image covered by the two binary cutout mask images. And subtracting the common area image from the binary cutout mask image of the aligned portrait image to obtain a mask image of a printing missing part, and determining whether a printing missing condition exists or not through comparative analysis of two extraction results, so that card witness portrait printing missing detection is automatically realized.

Description

Technical Field

[0001] The present invention relates to the technical field of card printing missing detection, and in particular to a card printing missing detection method and device. Background Art

[0002] In the automated card production process, accurate printing of the portrait is crucial. However, printing equipment may malfunction, sometimes resulting in partial loss of the portrait area after printing.

[0003] To ensure the quality of cards, it is usually necessary to manually check the printed cards to identify and remove those with incomplete portraits. This manual inspection method is not only time-consuming and labor-intensive, but also prone to missed inspections or misjudgments.

[0004] In addition, the limitations of the printing equipment itself cause color distortion between the printed portrait and the electronic original. At the same time, during the imaging process, the image color will be further distorted due to the influence of ambient light. These factors together increase the difficulty of automatically detecting missing portrait prints. In addition, the complex background patterns and anti-counterfeiting logos designed to improve the security and aesthetics of the card bring additional challenges to accurately identifying missing prints.

[0005] In view of this, it is urgent to develop an intelligent solution to achieve efficient detection of missing card portrait printing. Summary of the invention

[0006] In order to solve the technical problem of automatic detection of missing card portrait printing, the present invention provides a method and device for detecting missing card portrait printing. The following technical solution is adopted: A card image printing missing detection method comprises the following steps: Step 1, imaging the printed card image to obtain an image to be detected; Step 2, performing portrait detection on the electronically printed original image and the image to be detected, respectively, to obtain the portrait position of the electronically printed original image and the portrait position of the image to be detected; Step 3, cutting out the portrait area according to the portrait position of the image to be detected, and obtaining the portrait image to be detected; cutting out the portrait area according to the portrait position of the electronically printed original image, and obtaining the electronically printed portrait image; Step 4, extracting feature points from the electronically printed portrait image and the portrait image to be detected, and performing feature point matching to obtain matching point pairs; Step 5, calculating the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected based on the matching point pairs, performing affine transformation on the electronically printed portrait image, intercepting the portrait area of ​​the transformed image, and obtaining an aligned portrait image; Step 6, using the portrait cutout algorithm, respectively cut out the portrait image to be detected and the aligned portrait image to obtain their respective binary cutout mask images; Step 7, respectively perform an AND operation on the two binary cutout mask images to obtain a common area image, and subtract the common area image from the binary cutout mask image of the aligned portrait image to obtain a missing print mask image.

[0007] By adopting the above technical solution, firstly, an image of a printed card is obtained by an imaging device as an image to be detected; in order to improve the accuracy of automatic detection, imaging can be performed under controllable lighting conditions to avoid color distortion and deformation of the card image as much as possible. Portrait detection is performed in the electronic printed original image and the image to be detected to determine the position of the portrait. Subsequent feature point extraction and matching are limited to the target range of the portrait, reducing the influence of complex background on feature point matching. The portrait image to be detected is obtained by cropping, so that the subsequent portrait cutout is limited to the cropped image range, reducing the influence of complex background on the portrait cutout. Feature point matching is performed to obtain matching point pairs, which helps to achieve accurate alignment of the electronic printed original image of the portrait with the portrait image to be detected. The matching point pairs are used to calculate the affine transformation matrix from the electronic printed original image to the portrait image to be detected. The matrix is ​​applied to perform affine transformation on the electronic printed original image, and the portrait area is cut out from it to obtain the aligned portrait image. The aligned portrait image and the portrait image to be detected have the same resolution. At the same resolution, when performing portrait cutout, the cutout result can be simply used for missing detection.

[0008] The portrait cutout algorithm is used to process the portrait image to be detected and the aligned portrait image respectively to generate their own binary cutout mask images. By training a portrait cutout algorithm, the portrait area can be accurately cut out in the card image with complex background and anti-counterfeiting pattern. The two binary cutout mask images are logically ANDed to obtain the area image covered by the two. Then, the common area image is subtracted from the binary cutout mask image of the aligned portrait image to obtain the mask image of the missing print part. By comparing and analyzing the two extraction results, it is determined whether there is a missing print, thereby automatically realizing the missing print detection of the card portrait.

[0009] Optionally, perform portrait object detection by training a YOLO detection model.

[0010] Optionally, expand and crop outside the range of the YOLO detection box.

[0011] By adopting the above technical solution, since YOLO cannot obtain the target position accurate to the pixel, it can be appropriately expanded outside the range of the YOLO detection frame for cropping. Appropriate expansion is performed outside the range of the YOLO detection frame, such as expanding the target width around the detection frame by 1 / 10 to avoid the detection frame drifting and causing part of the portrait area to be cropped.

[0012] Optionally, in step 4, SURF or SIFT feature point detection and matching methods are used to extract feature points and match feature points.

[0013] By adopting the above technical solution, SURF or SIFT feature points can obtain relatively accurate matching point pairs, which is helpful to achieve accurate alignment of the electronic printed portrait image with the portrait image to be detected. In addition to SURF or SIFT, other feature point detection algorithms can also be used, and the specific algorithms are not limited here.

[0014] Optionally, in step 4, the facial key points can be located on the electronically printed portrait image and the portrait image to be detected, and all key points or some stable key points can be selected to form matching point pairs. The facial key point location algorithm can use SDM or other algorithms, and the specific algorithm is not limited here. The matching point pairs can select key points such as the corners of the eyes, corners of the mouth, nose, eyebrows, etc., or other key points. There is no restriction on the specific key points here either.

[0015] By adopting the above scheme, the facial key point positioning algorithm can obtain the accurate position of the facial key points, which is helpful to achieve accurate alignment between the electronically printed portrait image and the portrait image to be detected.

[0016] Optionally, in step 5, the method for calculating the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected is: Step 51, based on the matching point pairs, estimate the affine transformation matrix H0 using the RANSAC or PROSAC algorithm; Step 52, calculating the inner point of the matching point pair according to H0; Step 53: for the interior points, estimate the new affine transformation matrix H using the least squares method or the RANSAC or PROSAC algorithm.

[0017] The method of performing affine transformation on the electronically printed portrait is: for the point (x, y) in the electronically printed portrait, after affine transformation, its coordinates are (x', y') = H*(x, y) T , where * represents matrix multiplication.

[0018] Optionally, the specific method of performing AND operation on two binary matting mask images is: img_same = cv2.bitwise_and(bw_ref,bw_print).

[0019] Here cv2.bitwise_and is an opencv function. bw_ref and bw_print are the binary cutout mask images of the aligned portrait image and the portrait image to be detected, respectively, and img_same is the common area image. In the binary cutout mask image, the value of the portrait pixel is 255, and the value of the background pixel is 0.

[0020] The method of subtracting the common area image from the binary matting mask image of the aligned portrait image is: img_miss = bw_ref - img_same. Here, "-" is the matrix subtraction operator, and img_miss means the missing print mask image.

[0021] By adopting the above technical solution, the portrait cutout algorithm can obtain the precise position of the portrait and realize pixel-level segmentation. The binary cutout mask images of the aligned portrait image and the portrait image to be detected are ANDed to obtain a common area image, that is, an area where no missing prints appear. The common area image is subtracted from the binary cutout mask image of the aligned portrait image to obtain a missing print mask image. In the missing print mask image, a pixel with a value of 255 indicates that the pixel is a portrait pixel in the aligned portrait image, but a background pixel in the portrait image to be detected. Based on this, it can be determined that the pixel in the portrait image to be detected is missing print.

[0022] Optionally, after obtaining the missing print mask image in step 7, the missing position is accurately marked according to the missing print mask image. The pixels in the missing print mask image whose pixel values ​​are not 0 are taken as the missing positions.

[0023] A device for detecting missing prints of a person's portrait on a card, comprising an imaging device, a memory and a processor. The imaging device is used to image a printed card image. The memory stores an electronically printed original image and a detection program designed using a method for detecting missing prints of a person's portrait on a card. The memory communicates with the imaging device to exchange images to be detected. The processor is connected to the memory for communication, runs the detection program to detect the images to be detected, and outputs the detection results.

[0024] Optionally, a display is also included, which is communicatively connected to the processor, and the processor controls the display to display the detection results.

[0025] In summary, the present invention includes at least the following beneficial technical effects: The present invention can provide a card portrait printing missing detection method and device. First, an image of a printed card is obtained as an image to be detected by an imaging device; portrait detection is performed in the electronic printed original image and the image to be detected respectively to determine the position of the portrait. Subsequent feature point extraction and matching are limited to the portrait target range, reducing the influence of complex background on feature point matching. The portrait image to be detected is cropped to obtain a portrait image to be detected, so that the subsequent portrait cutout is limited to the cropped image range, reducing the influence of complex background on the portrait cutout. Feature point matching is performed to obtain matching point pairs, which is helpful to achieve accurate alignment of the electronic printed portrait image and the portrait image to be detected. The matching point pairs are used to calculate the affine transformation matrix from the electronic printed portrait image to the portrait image to be detected. The matrix is ​​applied to perform affine transformation on the electronic printed portrait image, and the portrait area is cut out therefrom to obtain the aligned portrait image. The aligned portrait image and the portrait image to be detected have the same resolution. The portrait cutout is performed at the same resolution. The portrait cutout algorithm is used to process the portrait image to be detected and the aligned portrait image respectively to generate their own binary cutout mask images. The two binary cutout mask images are logically ANDed to obtain the area image covered by both. The common area image is subtracted from the binary cutout mask image of the aligned portrait image to obtain the mask image of the missing print part. By comparing and analyzing the two extraction results, it is determined whether there is a missing print and accurately marks the missing position, thereby automatically realizing the missing print detection of the card portrait. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a card and document portrait printing missing detection method of the present invention; DETAILED DESCRIPTION

[0027] The present invention is further described in detail below with reference to the accompanying drawings. The embodiment of the present invention discloses a method and device for detecting missing print of a card or certificate portrait.

[0028] Reference Figure 1 A card image printing missing detection method comprises the following steps: Step 1, imaging the printed card image to obtain an image to be detected; Step 2, performing portrait detection on the electronically printed original image and the image to be detected, respectively, to obtain the portrait position of the electronically printed original image and the portrait position of the image to be detected; Step 3, cutting out the portrait area according to the portrait position of the image to be detected, and obtaining the portrait image to be detected; cutting out the portrait area according to the portrait position of the electronically printed original image, and obtaining the electronically printed portrait image; Step 4, extracting feature points from the electronically printed portrait image and the portrait image to be detected, and performing feature point matching to obtain matching point pairs; Step 5, calculating the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected based on the matching point pairs, performing affine transformation on the electronically printed portrait image, intercepting the portrait area of ​​the transformed image, and obtaining an aligned portrait image; Step 6, using the portrait cutout algorithm, respectively cut out the portrait image to be detected and the aligned portrait image to obtain their respective binary cutout mask images; Step 7, respectively perform an AND operation on the two binary cutout mask images to obtain a common area image, and subtract the common area image from the binary cutout mask image of the aligned portrait image to obtain a missing print mask image.

[0029] First, the image of the printed card is obtained by an imaging device as the image to be detected; in order to improve the accuracy of automatic detection, imaging can be performed under controllable lighting conditions to avoid color distortion and deformation of the card image as much as possible. Portrait detection is performed in the electronic printed original image and the image to be detected to determine the position of the portrait. Subsequent feature point extraction and matching are limited to the portrait target range to reduce the influence of complex background on feature point matching. The portrait image to be detected is cropped to limit the subsequent portrait cutout to the cropped image range to reduce the influence of complex background on the portrait cutout. Feature point matching is performed to obtain matching point pairs, which helps to achieve accurate alignment of the electronic printed portrait image with the portrait image to be detected. The matching point pairs are used to calculate the affine transformation matrix from the electronic printed portrait image to the portrait image to be detected. The matrix is ​​applied to perform affine transformation on the electronic printed portrait image, and the portrait area is cut out from it to obtain the aligned portrait image. The aligned portrait image and the portrait image to be detected have the same resolution. At the same resolution, the portrait cutout is performed, and the cutout result can be simply used for missing detection.

[0030] The portrait cutout algorithm is used to process the portrait image to be detected and the aligned portrait image respectively to generate their own binary cutout mask images. By training a portrait cutout algorithm, the portrait area can be accurately cut out in the card image with complex background and anti-counterfeiting pattern. The two binary cutout mask images are logically ANDed to obtain the area image covered by the two. Then, the common area image is subtracted from the binary cutout mask image of the aligned portrait image to obtain the mask image of the missing print part. By comparing and analyzing the two extraction results, it is determined whether there is a missing print, thereby automatically realizing the missing print detection of the card portrait.

[0031] Perform portrait target detection by training a YOLO detection model.

[0032] Expand and crop outside the range of the YOLO detection box.

[0033] Since YOLO cannot obtain the target position accurately to the pixel, it can be appropriately expanded outside the range of the YOLO detection frame for cropping. Appropriate expansion can be performed outside the range of the YOLO detection frame, such as expanding the target width around the detection frame by 1 / 10 to avoid the detection frame drifting and causing part of the portrait area to be cropped.

[0034] In step 4, SURF or SIFT feature point detection and matching methods are used to extract feature points and match feature points.

[0035] SURF or SIFT feature points can obtain relatively accurate matching point pairs, which helps to achieve accurate alignment between the original electronic print of the portrait and the portrait image to be detected.

[0036] In step 4, facial key points may be located on the electronically printed portrait image and the portrait image to be detected, and all key points or some stable key points may be selected to form matching point pairs.

[0037] By adopting the above scheme, the facial key point positioning algorithm can obtain the accurate position of the facial key points, which is helpful to achieve accurate alignment between the electronically printed portrait image and the portrait image to be detected.

[0038] In step 5, the method for calculating the affine transformation matrix from the electronically printed original image to the portrait image to be detected is: Step 51, based on the matching point pairs, estimate the affine transformation matrix H0 using the RANSAC or PROSAC algorithm; Step 52, calculating the inner point of the matching point pair according to H0; Step 53: for the interior points, estimate the new affine transformation matrix H using the least squares method or the RANSAC or PROSAC algorithm.

[0039] The method of affine transformation of the electronic print original image is: for the point (x, y) in the electronic print portrait, after affine transformation, its coordinates are (x', y') = H*(x, y) T , where * represents matrix multiplication.

[0040] The specific method of performing AND operation on two binary cutout mask images is: img_same = cv2.bitwise_and(bw_ref,bw_print).

[0041] Here cv2.bitwise_and is an opencv function. bw_ref and bw_print are the binary cutout mask images of the aligned portrait image and the portrait image to be detected, respectively, and img_same is the common area image. In the binary cutout mask image, the value of the portrait pixel is 255, and the value of the background pixel is 0.

[0042] The method of subtracting the common area image from the binary matting mask image of the aligned portrait image is: img_miss = bw_ref - img_same. Here, "-" is the matrix subtraction operator, and img_miss means the missing print mask image.

[0043] After obtaining the missing print mask image in step 7, the missing position is accurately marked according to the missing print mask image.

[0044] A device for detecting missing prints of a card or certificate portrait comprises an imaging device, a memory and a processor. The imaging device is used to image a printed card or certificate image. The memory stores an electronically printed original image and a detection program designed using a card or certificate portrait missing print detection method. The memory communicates with the imaging device to exchange images to be detected. The processor is connected to the memory for communication, runs the detection program to detect the images to be detected, and outputs the detection results.

[0045] It also includes a display, which is communicatively connected to the processor, and the processor controls the display to display the detection results.

[0046] The following uses a specific implementation case to illustrate the implementation principle of a card image printing missing detection method and device: Step 1: image the printed card image to obtain the image to be detected. In order to improve the accuracy of the algorithm, the front of the card needs to be photographed during the imaging process to prevent the introduction of perspective transformation.

[0047] Step 2: Perform portrait detection on the electronic print original image and the image to be detected respectively to obtain the portrait position of the electronic print original image and the portrait position of the image to be detected; and train a YOLO detector by collecting samples to detect portraits.

[0048] Step 3, cropping the portrait area according to the portrait position of the image to be detected to obtain the portrait image to be detected; cropping the portrait area according to the portrait position of the electronically printed original image to obtain the electronically printed portrait image; after cropping, the influence of the complex non-portrait area on the subsequent feature point matching and portrait clipping can be reduced.

[0049] Step 4, extracting SIFT feature points from the electronically printed portrait image and the portrait image to be detected respectively, and performing feature point matching to obtain matching point pairs; Step 5, based on the matching point pairs, calculate the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected. The affine transformation matrix can be calculated based on the findHomography function of opencv. Then, an affine transformation is performed on the electronically printed portrait image, and the portrait area of ​​the transformed image is intercepted to obtain an aligned portrait image. The aligned portrait image and the portrait image to be detected have the same resolution. Due to the influence of the complex background, there are some mismatches in the matching point pairs. In order to improve the accuracy of the affine transformation matrix estimation, first use RANSAC or PROSAC to estimate the affine transformation matrix, calculate the inliers, and then re-estimate the affine transformation matrix based on the least squares method and the inliers. According to this method, an accurate affine transformation matrix can be obtained.

[0050] Step 6: Use the portrait cutout algorithm to cut out the portrait image to be detected and the aligned portrait image respectively to obtain their respective binary cutout mask images. The portrait cutout algorithm is trained based on a large number of samples and is still robust to complex backgrounds.

[0051] Step 7, respectively perform an AND operation on the two binary cutout mask images to obtain a common area image, and subtract the common area image from the binary cutout mask image of the aligned portrait image to obtain a missing print mask image.

[0052] Pixels that are not 0 in the missing mask image are determined to be missing printing pixels, and the missing printing result is obtained accordingly.

[0053] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A card image printing missing detection method, characterized in that: The following steps are involved: Step 1, imaging the printed card image to obtain an image to be detected; Step 2, performing portrait detection on the electronically printed original image and the image to be detected, respectively, to obtain the portrait position of the electronically printed original image and the portrait position of the image to be detected; Step 3, cutting out the portrait area according to the portrait position of the image to be detected, and obtaining the portrait image to be detected; cutting out the portrait area according to the portrait position of the electronically printed original image, and obtaining the electronically printed portrait image; Step 4, extracting feature points from the electronically printed portrait image and the portrait image to be detected, and performing feature point matching to obtain matching point pairs; Step 5, calculating the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected based on the matching point pairs, performing affine transformation on the electronically printed portrait image, intercepting the portrait area of ​​the transformed image, and obtaining an aligned portrait image; Step 6, using the portrait cutout algorithm, respectively cut out the portrait image to be detected and the aligned portrait image to obtain their respective binary cutout mask images; Step 7, respectively perform an AND operation on the two binary cutout mask images to obtain a common area image, and subtract the common area image from the binary cutout mask image of the aligned portrait image to obtain a missing print mask image.

2. A card image printing missing detection method according to claim 1, characterized in that: In step 2, a YOLO detection model is trained to detect portrait targets.

3. A card image printing missing detection method according to claim 2, characterized in that: In step 3, the image is expanded and cropped outside the range of the YOLO detection frame.

4. A card image printing missing detection method according to claim 1, characterized in that: In step 4, SURF or SIFT feature point detection and matching methods are used to extract feature points and match feature points.

5. The card image printing missing detection method according to claim 1, characterized in that: In step 4, facial key points are located on the electronically printed portrait image and the portrait image to be detected, and all key points or some stable key points are selected to form matching point pairs.

6. A card image printing missing detection method according to claim 1, characterized in that: In step 5, the method for calculating the affine transformation matrix from the electronically printed portrait image to the portrait image to be detected is: Step 51, based on the matching point pairs, estimate the affine transformation matrix H0 using the RANSAC or PROSAC algorithm; Step 52, calculating the inner point of the matching point pair according to H0; Step 53, for the interior points, estimate a new affine transformation matrix H using the least squares method or the RANSAC or PROSAC algorithm; The method of performing affine transformation on the electronically printed portrait is: for the point (x, y) in the electronically printed portrait, after affine transformation, its coordinates are (x', y') = H*(x, y) T , where * represents matrix multiplication.

7. A card image printing missing detection method according to claim 1, characterized in that: In step 7, the specific method of performing AND operation on two binary cutout mask images is: img_same = cv2.bitwise_and(bw_ref,bw_print); cv2.bitwise_and is an opencv function, bw_ref and bw_print are the binary cutout mask images of the aligned portrait image and the portrait image to be detected, respectively, and img_same is the common area image. In the binary cutout mask image, the value of the portrait pixel is 255, and the value of the background pixel is 0; The method of subtracting the common area image from the binary matting mask image of the aligned portrait image is: img_miss=bw_ref-img_same; Among them, - is the matrix subtraction operator, and img_miss means missing printing mask image.

8. The card image printing missing detection method according to claim 1, characterized in that: After obtaining the missing print mask image in step 7, the missing position is accurately marked according to the missing print mask image.

9. A card image printing missing detection device, characterized in that: The invention comprises an imaging device, a memory and a processor, wherein the imaging device is used to image the printed card image, the memory stores the electronic printed original image and a detection program designed by using the card image missing printing detection method according to any one of claims 1 to 8, the memory communicates with the imaging device to exchange the image to be detected, the processor is connected to the memory for communication, runs the detection program to detect the image to be detected, and outputs the detection result.

10. The card image printing missing detection device according to claim 9, characterized in that: It also includes a display, which is communicatively connected to the processor, and the processor controls the display to display the detection results.

Images