Quadrilateral distortion correction method based on close-range photographing

Abstract

The invention relates to the technical field of video imaging, and discloses a quadrilateral distortion correction method based on close-range photographing, which comprises the following steps: detecting the position of a certificate; performing preliminary perspective correction; calculating a distortion boundary; calculating a correction boundary; constructing a distortion correction mapping relation; distortion correction is carried out; performing fine adjustment perspective correction; matting the image; the certificate position detection comprises the following steps: firstly, detecting a quadrilateral shape similar to a card in a picture by using a deep learning model; the quadrilateral distortion correction method based on close-range photographing can be used for correcting linear edge distortion caused by close-range photographing of card certificates by different mobile phones, so that the original linear edge effect of the card is recovered; perspective correction can be carried out on cards and certificates shot at different angles on the basis of distortion correction, so that the corrected card pictures better conform to the top view effect of the actual size; the corrected card type certificate can be subjected to matting under the condition that the fillet characteristics of the card are kept.

Description

Technical Field

[0001] The invention relates to the technical field of video imaging, and in particular to a method for correcting quadrilateral distortion based on close-range photography. Background Art

[0002] Since some existing business processes require users to provide some card certificates (such as ID cards, bank cards) and other photo screenshots to upload to the system, some users will use mobile phones to take pictures at a relatively close distance in order to take clearer pictures. The close-range shooting process will cause the straight edges of the cards to deform and become slightly curved. The screenshot cannot use the straight line screenshot method to retain the original shape of the card certificates. Most of the existing distortion corrections are aimed at keeping the camera parameters unchanged, and use a checkerboard to correct fisheye wide-angle distortion. However, for the above business scenarios, the parameters of each user's mobile phone camera are different, and it is impossible to correct the distortion caused by close-range photography according to the method of keeping the parameters unchanged. Therefore, the present invention proposes a quadrilateral distortion correction method for close-range photography with different mobile phones. Summary of the invention

[0003] The purpose of the present invention is to provide a method for correcting quadrilateral distortion based on close-range photography to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a method for correcting quadrilateral distortion based on close-up photography, comprising the following steps: detecting the position of the document; preliminary perspective correction; calculating the distortion boundary; calculating the correction boundary; constructing a distortion correction mapping relationship; distortion correction; fine-tuning the perspective correction; cutting out the image;

[0005] The method of detecting the position of the certificate is as follows: firstly, a deep learning model is used to detect a quadrilateral shape similar to the card in the image;

[0006] The preliminary perspective correction: after finding the card, use the quadrilateral information provided by the model to perform preliminary perspective transformation correction on the card image;

[0007] The calculation of the distortion boundary: finding multiple points on each distorted edge of the card that together constitute the distortion boundary;

[0008] The calculation of the correction boundary: calculating the correction boundary after correction by analyzing the points on the distortion boundary;

[0009] The distortion correction mapping relationship is constructed by making a one-to-one correspondence between points on the distortion boundary and points on the correction boundary, and calculating the correction mapping relationship according to the correspondence relationship;

[0010] The distortion correction: performing distortion correction on the initially corrected image according to the calculated correction mapping relationship;

[0011] The fine-tuning perspective correction: further fine-tuning the perspective transformation correction of the image according to the calculated correction boundary;

[0012] The cutout process includes making a card rounded corner mask and cutting out the card-type certificate.

[0013] Preferably, the detection certificate location is specifically:

[0014] a. First, collect various card ID images with edge distortion taken by mobile phones at close range, and perform quadrilateral annotation on the collected images;

[0015] b. Use deep learning to train a quadrilateral object detection model on the labeled data;

[0016] c. Input the image to be distorted into the quadrilateral detection model to obtain the original coordinate information [P0, P1, P2, P3] of the four vertices of the quadrilateral where the card in the image is located, which contains the horizontal coordinate x and vertical coordinate y information of each point.

[0017] Preferably, the initial perspective correction is specifically:

[0018] a. Calculate the coordinate information of the four vertices [P0', P1', P2', P3'] after converting the original quadrilateral into a rectangle through the original vertex coordinate information [P0, P1, P2, P3]. The calculation method of [P0', P1', P2', P3'] is as follows:

[0019] a). Calculate the lengths of the top, bottom, left and right sides based on the above four points [P0, P1, P2, P3], and record them as: top_L, btm_L, left_L, right_L;

[0020] b). The calculation method of the rectangle width w and height h after perspective correction is as follows:

[0021] w=int((top_L+btm_L) / 2)

[0022] h = int((left_L+right_L) / 2)

[0023] c). Define the thresholds scale_w and scale_h, which are used to control the proportion of the area after perspective correction based on the original w and h to be retained or cut. The default value is 1.0. The specific calculation order of the 4 points after perspective correction is upper left->upper right->lower right->lower left. The coordinates are marked as P0'[x1, y1], P1'[x2, y1], P2'[x2, y2], P3'[x1, y2], and the calculation method of x1, y1, x2, y2 is as follows:

[0024] x1 = w*(1-1 / scale_w) / 2;

[0025] y1 = h*(1-1 / scale_h) / 2;

[0026] x2=w-x1;

[0027] y2=h-y1;

[0028] b. Calculate the perspective transformation matrix M through the transformation of [P0, P1, P2, P3]-->[P0', P1', P2', P3'];

[0029] c. Perform a preliminary perspective correction operation on the image Image through the transformation matrix M to obtain Image1.

[0030] Preferably, the distortion boundary is calculated as follows:

[0031] a. The four edges formed by [P0', P1', P2', P3'] in clockwise direction are recorded as Lines[L0, L1, L2, L3], and four empty sets Edges[E0, E1, E2, E3] are initialized for the four edges, which are used to store the point sets on each distorted boundary corresponding to Lines;

[0032] b. Take each side in Lines as the center line, expand a smaller range to both sides to obtain the corresponding 4 strip-shaped rectangular areas, and perform the following operations on each strip-shaped rectangular area:

[0033] a). Get the corresponding part of the current area in Image1 and copy it to get a rectangular area image box_image; convert box_image into a grayscale image gray_box_image, and perform image smoothing on it to reduce the noise in the image, and get the grayscale image gray_box_image after smoothing;

[0034] b). Use the edge detection algorithm on gray_box_image to obtain its corresponding edge detection image canny_box_image, set a minimum straight line segment length threshold L, use Hough line detection to detect straight line segments on canny_box_image, and filter out smaller straight line segments through the threshold L. After the remaining straight line segments, take the two endpoints of each straight line segment respectively to obtain a set of 2N points (N is the number of remaining straight line segments), and put these 2N points into the empty set of the position in Edges corresponding to the current area.

[0035] Preferably, the calculation of the correction boundary is specifically as follows:

[0036] Each set in the calculated Edges[E0, E1, E2, E3] corresponds to the set of points on each distorted edge. Initialize an empty set FunctionEdges[] for storing the straight line where the correction boundary is located, traverse each edge point set Ei in Edges and perform the following operations (i∈[0, 1, 2, 3]):

[0037] a). Determine whether the boundary to which Ei belongs is horizontal or vertical. If Ei is a set of points in the horizontal direction, sort the points in Ei from left to right along the X-axis of the points; if Ei is in the vertical direction, sort the points in Ei from top to bottom along the Y-axis of the points;

[0038] b). According to the current sorted order of the Ei point set, find the two points pN-1 and pN located at the N-1 and N positions of the two middle points;

[0039] c). Establish a straight line model: fi(x)=wix+b, where wi and bi are the straight line model coefficients. Mark the coordinates corresponding to the two points pN-1 and pN as (xn-1, yn-1), (xn, yn) and input them into the straight line model to calculate the coefficients wi and bi;

[0040] d). Take the first and last two points p0 and p2N-1 in Ei, calculate the distances from these two points to the line fi(x) respectively and take the maximum distance as distance, translate the line model along the point with the maximum distance by distance / 2 to obtain a new line model, bi' is the coefficient after translation, the new line model: fi(x) = wix + bi', the line where this line model is located is the correction boundary corresponding to the current distortion boundary, put it into the correction boundary set FunctionEdges.

[0041] Preferably, the distortion correction mapping relationship is constructed as follows:

[0042] The distortion boundary point set Edges[E0, E1, E2, E3] and the line set FunctionEdges[f0(x), f1(x), f2(x), f3(x)] to which the correction boundary belongs initialize a set of points CorrectEdges[E'0, E'1, E'2, E'3] on the correction boundary, where E'0, E'1, E'2, E'3 are all empty sets, which are used to store the points on the correction boundary corresponding to each point on the distortion boundary. Now the following operations are needed to fill E'0, E'1, E'2, E'3:

[0043] a. Synchronously traverse the elements in the distortion boundary Edges and the correction boundary FunctionEdges, record the distortion boundary traversed synchronously as Ei, record the correction boundary line as fi(x)=wix+bi', and then continue to traverse each point in Ei as pij;

[0044] b. Draw a perpendicular line from point pij to the line fi(x), and calculate the intersection point p'ij of the perpendicular line and fi(x). This intersection point is the point pij on the distortion boundary Ei corresponding to the point on the correction boundary fi(x). Put point p'ij into the set E'i.

[0045] Preferably, the distortion correction is specifically:

[0046] The point set Edges of the distorted boundary and the point set CorrectEdges on the corresponding corrected boundary have been calculated, as follows:

[0047] Edges=[E0, E1, E2, E3], Ei=[pi0, pi1,..., pij];

[0048] CorrectEdges=[E'0, E'1, E'2, E'3], E'i=[p'i0, p'i1,..., p'ij];

[0049] By transforming the corresponding points of Edges->CorrectEdges, the distortion correction matrix M1 is calculated. By applying the distortion correction matrix M1 to the image Image1, the distortion-corrected image Image2 can be obtained.

[0050] Preferably, the fine-tuning perspective correction is specifically as follows: since the quadrilateral detection of the external tangent of the distortion boundary of the previous image is completed by model detection, there will be errors in the model detection, resulting in errors in the image after perspective correction. The correction boundary FunctionEdges[f0(x), f1(x), f2(x), f3(x)] can further reduce this error, so it is necessary to use the correction boundary to further perform fine-tuning perspective correction. The specific steps are as follows:

[0051] a. Calculate the four intersection points of the four lines f0(x), f1(x), f2(x), and f3(x) in the order of upper left -> upper right -> lower right -> lower left, recorded as intetsection_p0, intetsection_p1, intetsection_p2, intetsection_p3;

[0052] b. Calculate the four points after perspective correction based on the above four intersection points, in the order of upper left -> upper right -> lower right -> lower left, recorded as pespective_p0, pespective_p1, pespective_p2, pespective_p3;

[0053] c. According to the transformation of intetsection_pi->pespective_pi, i∈[0, 1, 2, 3], calculate the new perspective transformation matrix M2, and use the perspective transformation matrix M2 to fine-tune the perspective correction of Image2 to obtain Image3.

[0054] Preferably, the cutout is specifically:

[0055] a. Since the ID images uploaded by taking photos generally have a unified target size requirement, it is necessary to scale the image Image3 to the target size;

[0056] b. Select a standard card document image at the target size as the template image, then use the segment annotation tool to mark the four rounded corners in the image, and use the marked rounded corners as the rounded corner template and record them as mask;

[0057] c. Use alpha image fusion technology to process the scaled Image3 and the mask template to complete the final cutout containing rounded corner information and place the cutout part on the specified background.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The quadrilateral distortion correction method based on close-range photography can be used to correct the straight line distortion caused by close-range photography of card-type certificates with different mobile phones, so as to restore the original straight line effect of the card; on the basis of distortion correction, perspective correction can be performed on card-type certificates photographed at different angles, so that the corrected card image is more in line with the top view effect of the actual size; the corrected card-type certificates can be cut out while retaining the rounded corner features of the card.

[0060] 2. This method of correcting quadrilateral distortion based on close-up photography can be applied to some real-life situations where a mobile phone is used to take photos of identity cards, residence permits, bank cards, etc. and upload them to the system to generate scanned file formats, which can save the step of using a printer and is more convenient and concise; it can also be used for the distortion correction of other objects containing quadrilateral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the distortion correction process of the present invention;

[0062] Figure 2This is the original image of the card before distortion correction;

[0063] Figure 3 This is the effect picture after card distortion correction. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] See also Figures 1 to 3 In an embodiment of the present invention, a method for correcting quadrilateral distortion based on close-up photography includes the following steps: detecting the position of a document; performing preliminary perspective correction; calculating a distortion boundary; calculating a correction boundary; constructing a distortion correction mapping relationship; performing distortion correction; fine-tuning perspective correction; cutting out the image;

[0066] It can be used to correct the straight line distortion caused by close-up photography of card-like documents with different mobile phones, so as to restore the original straight line effect of the card;

[0067] Based on distortion correction, perspective correction can be performed on cards and ID cards taken at different angles, so that the corrected card images are more consistent with the top view effect of the actual size;

[0068] The corrected card-type ID document can be cut out while retaining the rounded corner features of the card.

[0069] Detecting the location of the ID card: First, a deep learning model is used to detect a quadrilateral shape similar to the card in the image;

[0070] Preliminary perspective correction: After finding the card, use the quadrilateral information provided by the model to perform preliminary perspective transformation correction on the card image, so that the edge of the card image looks more like a rectangle that conforms to the actual shape, rather than a distorted quadrilateral, while ignoring edge distortion;

[0071] Calculate the distortion boundary: find multiple points on each distorted edge of the card that together constitute the distortion boundary;

[0072] Calculate the correction boundary: Calculate the corrected correction boundary by analyzing the points on the distortion boundary;

[0073] Construct the distortion correction mapping relationship: correspond the points on the distortion boundary to the points on the correction boundary one by one, and calculate the correction mapping relationship based on the correspondence relationship;

[0074] Distortion correction: The initially corrected image is subjected to distortion correction according to the calculated correction mapping relationship;

[0075] Fine-tuning perspective correction: further fine-tuning the perspective transformation correction of the image based on the calculated correction boundary;

[0076] Cutout: Create a card-shaped rounded corner mask to cut out card-type documents.

[0077] It can be applied to some real-life situations where you can take photos of identity cards, residence permits, bank cards, etc. with a mobile phone and upload them to the system to generate scanned file formats, which can save the step of using a printer and is more convenient and simple. It can also be used for distortion correction of other images containing quadrilateral objects.

[0078] As a further implementation scheme of the above invention: the position of the detection document is specifically:

[0079] a. First, collect various card ID images with distorted edges taken by mobile phones at close range, and annotate the collected images with quadrilaterals. Because the edges of the cards in the images are distorted and not straight lines, each edge needs to be annotated along the tangent line of the outermost edge of the distorted edge;

[0080] b. Use deep learning to train a quadrilateral object detection model on the labeled data;

[0081] c. Input the image to be distorted into the quadrilateral detection model to obtain the original coordinate information [P0, P1, P2, P3] of the four vertices of the quadrilateral where the card in the image is located, which contains the horizontal coordinate x and vertical coordinate y information of each point.

[0082] As a further embodiment of the above invention: the preliminary perspective correction is specifically:

[0083] a. Calculate the coordinate information of the four vertices [P0', P1', P2', P3'] after converting the original quadrilateral into a rectangle through the original vertex coordinate information [P0, P1, P2, P3]. The calculation method of [P0', P1', P2', P3'] is as follows:

[0084] a). Calculate the lengths of the top, bottom, left and right sides based on the above four points [P0, P1, P2, P3], and record them as: top_L, btm_L, left_L, right_L;

[0085] b). The calculation method of the rectangle width w and height h after perspective correction is as follows:

[0086] w=int((top_L+btm_L) / 2)

[0087] h = int((left_L+right_L) / 2)

[0088] Note: int() represents rounding operation.

[0089] c). Define the thresholds scale_w and scale_h, which are used to control the proportion of the area after perspective correction based on the original w and h to be retained or cut. The default value is 1.0, and the value can be flexibly adjusted according to your needs. If it is greater than 1.0, it means that after perspective correction, some background areas will be retained in addition to the card edge. If it is less than 1.0, it means that a certain degree of cutting will be performed on the card edge after perspective correction. The specific calculation order of the 4 points after perspective correction is upper left->upper right->lower right->lower left. The coordinates are marked as P0'[x1, y1], P1'[x2, y1], P2'[x2, y2], P3'[x1, y2], and the calculation method of x1, y1, x2, y2 is as follows:

[0090] x1 = w*(1-1 / scale_w) / 2;

[0091] y1 = h*(1-1 / scale_h) / 2;

[0092] x2=w-x1;

[0093] y2=h-y1;

[0094] b. Calculate the perspective transformation matrix M through the transformation of [P0, P1, P2, P3]-->[P0', P1', P2', P3'];

[0095] c. Perform a preliminary perspective correction operation on the image Image through the transformation matrix M to obtain Image1.

[0096] As a further implementation of the above invention: calculating the distortion boundary is specifically as follows:

[0097] a. The four edges formed by [P0', P1', P2', P3'] in clockwise direction are recorded as Lines[L0, L1, L2, L3], and four empty sets Edges[E0, E1, E2, E3] are initialized for the four edges, which are used to store the point sets on each distorted boundary corresponding to Lines;

[0098] b. Take each edge in Lines as the center line and expand a smaller range to both sides to obtain the corresponding four strip-shaped rectangular areas. The expansion range is generally set based on actual experience so that the entire distorted edge can be included in this range. Perform the following operations on each strip-shaped rectangular area:

[0099] a). Get the corresponding part of the current area in Image1 and copy it to get a rectangular area image box_image; convert box_image into a grayscale image gray_box_image, and perform image smoothing on it to reduce the noise in the image, and get the grayscale image gray_box_image after smoothing;

[0100] b). Use the edge detection algorithm on gray_box_image to obtain its corresponding edge detection image canny_box_image, set a minimum straight line segment length threshold L, use Hough line detection to detect straight line segments on canny_box_image, and filter out smaller straight line segments through the threshold L. After the remaining straight line segments, take the two endpoints of each straight line segment respectively to obtain a set of 2N points (N is the number of remaining straight line segments), and put these 2N points into the empty set of the position in Edges corresponding to the current area.

[0101] As a further implementation of the above invention: calculating the correction boundary is specifically as follows:

[0102] Each set in the calculated Edges[E0, E1, E2, E3] corresponds to the set of points on each distorted edge. Initialize an empty set FunctionEdges[] for storing the straight line where the correction boundary is located, traverse each edge point set Ei in Edges and perform the following operations (i∈[0, 1, 2, 3]):

[0103] a). Determine whether the boundary to which Ei belongs is horizontal or vertical. If Ei is a set of points in the horizontal direction, sort the points in Ei from left to right along the X-axis of the points; if Ei is in the vertical direction, sort the points in Ei from top to bottom along the Y-axis of the points;

[0104] b). According to the current sorted order of the Ei point set, find the two points pN-1 and pN located at the N-1 and N positions of the two middle points. Note: the point index starts from 0;

[0105] c). Establish a straight line model: fi(x)=wix+b, where wi and bi are the straight line model coefficients. Mark the coordinates corresponding to the two points pN-1 and pN as (xn-1, yn-1), (xn, yn) and input them into the straight line model to calculate the coefficients wi and bi;

[0106] d). Take the first and last two points p0 and p2N-1 in Ei, calculate the distances from these two points to the line fi(x) respectively and take the maximum distance as distance, translate the line model along the point with the maximum distance by distance / 2 to obtain a new line model, bi' is the coefficient after translation, the new line model: fi(x) = wix + bi', the line where this line model is located is the correction boundary corresponding to the current distortion boundary, put it into the correction boundary set FunctionEdges.

[0107] As a further implementation of the above invention: constructing the distortion correction mapping relationship is specifically as follows:

[0108] The distortion boundary point set Edges[E0, E1, E2, E3] and the line set FunctionEdges[f0(x), f1(x), f2(x), f3(x)] to which the correction boundary belongs initialize a set of points CorrectEdges[E'0, E'1, E'2, E'3] on the correction boundary, where E'0, E'1, E'2, E'3 are all empty sets, which are used to store the points on the correction boundary corresponding to each point on the distortion boundary. Now the following operations are needed to fill E'0, E'1, E'2, E'3:

[0109] a. Synchronously traverse the elements in the distortion boundary Edges and the correction boundary FunctionEdges, record the distortion boundary traversed synchronously as Ei, record the correction boundary line as fi(x)=wix+bi', and then continue to traverse each point in Ei as pij;

[0110] b. Draw a perpendicular line from point pij to the line fi(x), and calculate the intersection point p'ij of the perpendicular line and fi(x). This intersection point is the point pij on the distortion boundary Ei corresponding to the point on the correction boundary fi(x). Put point p'ij into the set E'i.

[0111] As a further embodiment of the above invention: the distortion correction is specifically as follows:

[0112] The point set Edges of the distorted boundary and the point set CorrectEdges on the corresponding corrected boundary have been calculated, as follows:

[0113] Edges=[E0, E1, E2, E3], Ei=[pi0, pi1,..., pij];

[0114] CorrectEdges=[E'0, E'1, E'2, E'3], E'i=[p'i0, p'i1,..., p'ij];

[0115] Note: i∈[0, 1, 2, 3], j is the point index on the current distortion boundary, which is determined by the number of detected straight line segments

[0116] By transforming the corresponding points of Edges->CorrectEdges, the distortion correction matrix M1 is calculated. By applying the distortion correction matrix M1 to the image Image1, the distortion-corrected image Image2 can be obtained.

[0117] As a further implementation scheme of the above invention: fine-tuning perspective correction is specifically as follows: since the quadrilateral detection of the external tangent of the distortion boundary of the previous image is completed by model detection, there will be errors in the model detection, resulting in errors in the image after perspective correction. The correction boundary FunctionEdges[f0(x), f1(x), f2(x), f3(x)] can further reduce this error. Therefore, it is necessary to use the correction boundary to further perform fine-tuning perspective correction. The specific steps are as follows:

[0118] a. Calculate the four intersection points of the four lines f0(x), f1(x), f2(x), and f3(x) in the order of upper left -> upper right -> lower right -> lower left, recorded as intetsection_p0, intetsection_p1, intetsection_p2, intetsection_p3. The calculation method of the intersection points of the lines is a conventional operation and will not be described in detail.

[0119] b. Calculate the four points after perspective correction based on the above four intersection points (the calculation method is the same as the steps in the preliminary perspective correction, scale_w and scale_h are both set to 1.0, and no further details are given here). The order is upper left -> upper right -> lower right -> lower left, recorded as pespective_p0, pespective_p1, pespective_p2, pespective_p3;

[0120] c. According to the transformation of intetsection_pi->pespective_pi, i∈[0, 1, 2, 3], calculate the new perspective transformation matrix M2, and use the perspective transformation matrix M2 to fine-tune the perspective correction of Image2 to obtain Image3.

[0121] As a further implementation scheme of the above invention: the cutout is specifically as follows:

[0122] a. Since the ID images uploaded by taking photos generally have a unified target size requirement, it is necessary to scale the image Image3 to the target size;

[0123] b. Select a standard card document image at the target size as the template image, then use the segment annotation tool to mark the four rounded corners in the image, and use the marked rounded corners as the rounded corner template and record them as mask;

[0124] c. Use alpha image fusion technology to process the scaled Image3 and the mask template to complete the final cutout containing rounded corner information and place the cutout part on the specified background (such as A4 paper size).

[0125] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for correcting quadrilateral distortion based on close-up photography, characterized in that: The following steps are involved: Detect the document position; perform preliminary perspective correction; calculate the distortion boundary; calculate the correction boundary; Construct distortion correction mapping relationship; Distortion correction; Fine-tune perspective correction; Cutout; The method of detecting the position of the certificate is as follows: firstly, a deep learning model is used to detect a quadrilateral shape similar to the card in the image; The preliminary perspective correction: after finding the card, use the quadrilateral information provided by the model to perform preliminary perspective transformation correction on the card image; The calculation of the distortion boundary: finding multiple points on each distorted edge of the card that together constitute the distortion boundary; The calculation of the correction boundary: calculating the correction boundary after correction by analyzing the points on the distortion boundary; The distortion correction mapping relationship is constructed by making a one-to-one correspondence between points on the distortion boundary and points on the correction boundary, and calculating the correction mapping relationship according to the correspondence relationship; The distortion correction: performing distortion correction on the initially corrected image according to the calculated correction mapping relationship; The fine-tuning perspective correction: further fine-tuning the perspective transformation correction of the image according to the calculated correction boundary; The cutout process includes making a card rounded corner mask and cutting out the card-type certificate.

2. The method for correcting quadrilateral distortion based on close-range photography according to claim 1, characterized in that: The specific locations of the test certificates are: a. First, collect various card ID images with edge distortion taken by mobile phones at close range, and perform quadrilateral annotation on the collected images; b. Use deep learning to train a quadrilateral object detection model on the labeled data; c. Input the image to be distorted into the quadrilateral detection model to obtain the original coordinate information [P0, P1, P2, P3] of the four vertices of the quadrilateral where the card in the image is located, which contains the horizontal coordinate x and vertical coordinate y information of each point.

3. The method for correcting quadrilateral distortion based on close-up photography according to claim 2, characterized in that: Perspective correction is as follows: a. Calculate the coordinate information of the four vertices [P0', P1', P2', P3'] after converting the original quadrilateral into a rectangle through the original vertex coordinate information [P0, P1, P2, P3]. The calculation method of [P0', P1', P2', P3'] is as follows: a). Calculate the lengths of the top, bottom, left and right sides based on the above four points [P0, P1, P2, P3], and record them as: top_L, btm_L, left_L, right_L; b). The calculation method of the rectangle width w and height h after perspective correction is as follows: w=int((top_L+btm_L) / 2) h = int((left_L+right_L) / 2) c). Define the thresholds scale_w and scale_h, which are used to control the proportion of the area after perspective correction based on the original w and h to be retained or cut. The default value is 1.

0. The specific calculation order of the 4 points after perspective correction is upper left->upper right->lower right->lower left. The coordinates are marked as P0'[x1, y1], P1'[x2, y1], P2'[x2, y2], P3'[x1, y2], and the calculation method of x1, y1, x2, y2 is as follows: x1 = w*(1-1 / scale_w) / 2; y1 = h*(1-1 / scale_h) / 2; x2=w-x1; y2=h-y1; b. Calculate the perspective transformation matrix M through the transformation of [P0, P1, P2, P3]-->[P0', P1', P2', P3']; c. Perform a preliminary perspective correction operation on the image Image through the transformation matrix M to obtain Image1.

4. The method for correcting quadrilateral distortion based on close-range photography according to claim 3, characterized in that: The calculation of the distortion boundary is specifically: a. The four edges formed by [P0', P1', P2', P3'] in clockwise direction are recorded as Lines[L0, L1, L2, L3], and four empty sets Edges[E0, E1, E2, E3] are initialized for the four edges, which are used to store the point sets on each distorted boundary corresponding to Lines; b. Take each side in Lines as the center line, expand a smaller range to both sides to obtain the corresponding 4 strip-shaped rectangular areas, and perform the following operations on each strip-shaped rectangular area: a). Get the corresponding part of the current area in Image1 and copy it to get a rectangular area image box_image; convert box_image into a grayscale image gray_box_image, and perform image smoothing on it to reduce the noise in the image, and get the grayscale image gray_box_image after smoothing; b). Use the edge detection algorithm on gray_box_image to obtain its corresponding edge detection image canny_box_image, set a minimum straight line segment length threshold L, use Hough line detection to detect straight line segments on canny_box_image, and filter out smaller straight line segments through the threshold L. After the remaining straight line segments, take the two endpoints of each straight line segment respectively to obtain a set of 2N points (N is the number of remaining straight line segments), and put these 2N points into the empty set of the position in Edges corresponding to the current area.

5. The method for correcting quadrilateral distortion based on close-range photography according to claim 4, characterized in that: The calculation of the correction boundary is as follows: Each set in the calculated Edges[E0, E1, E2, E3] corresponds to the set of points on each distorted edge. Initialize an empty set FunctionEdges[] for storing the straight line where the correction boundary is located, traverse each edge point set Ei in Edges and perform the following operations (i∈[0, 1, 2, 3]): a). Determine whether the boundary to which Ei belongs is horizontal or vertical. If Ei is a set of points in the horizontal direction, sort the points in Ei from left to right along the X-axis of the points; If Ei is in the vertical direction, sort the point set in Ei from top to bottom along the Y axis of the point; b). According to the current sorted order of the Ei point set, find the two points pN-1 and pN located at the N-1 and N positions of the two middle points; c). Establish a straight line model: fi(x)=wix+b, where wi and bi are the straight line model coefficients. Mark the coordinates corresponding to the two points pN-1 and pN as (xn-1, yn-1), (xn, yn) and input them into the straight line model to calculate the coefficients wi and bi; d). Take the first and last two points p0 and p2N-1 in Ei, calculate the distances from these two points to the line fi(x) respectively and take the maximum distance as distance, translate the line model along the point with the maximum distance by distance / 2 to obtain a new line model, bi' is the coefficient after translation, the new line model: fi(x) = wix + bi', the line where this line model is located is the correction boundary corresponding to the current distortion boundary, put it into the correction boundary set FunctionEdges.

6. The method for correcting quadrilateral distortion based on close-range photography according to claim 5, characterized in that: The specific relationship for constructing the distortion correction mapping is: The distortion boundary point set Edges[E0, E1, E2, E3] and the line set FunctionEdges[f0(x), f1(x), f2(x), f3(x)] to which the correction boundary belongs initialize a set of points CorrectEdges[E'0, E'1, E'2, E'3] on the correction boundary, where E'0, E'1, E'2, E'3 are all empty sets, which are used to store the points on the correction boundary corresponding to each point on the distortion boundary. Now the following operations are needed to fill E'0, E'1, E'2, E'3: a. Synchronously traverse the elements in the distortion boundary Edges and the correction boundary FunctionEdges, record the distortion boundary traversed synchronously as Ei, record the correction boundary line as fi(x)=wix+bi', and then continue to traverse each point in Ei as pij; b. Draw a perpendicular line from point pij to the line fi(x), and calculate the intersection point p'ij of the perpendicular line and fi(x). This intersection point is the point pij on the distortion boundary Ei corresponding to the point on the correction boundary fi(x). Put point p'ij into the set E'i.

7. The method for correcting quadrilateral distortion based on close-range photography according to claim 6, characterized in that: Distortion correction is as follows: The point set Edges of the distorted boundary and the point set CorrectEdges on the corresponding corrected boundary have been calculated, as follows: Edges=[E0, E1, E2, E3], Ei=[pi0, pi1,..., pij]; CorrectEdges=[E'0, E'1, E'2, E'3], E'i=[p'i0, p'i1,..., p'ij]; By transforming the corresponding points of Edges->CorrectEdges, the distortion correction matrix M1 is calculated. By applying the distortion correction matrix M1 to the image Image1, the distortion-corrected image Image2 can be obtained.

8. The method for correcting quadrilateral distortion based on close-range photography according to claim 7, characterized in that: The specific steps of fine-tuning perspective correction are as follows: Since the quadrilateral detection of the external tangent of the distortion boundary of the previous image is completed by model detection, there will be errors in the model detection, which will cause errors in the image after perspective correction. The correction boundary FunctionEdges[f0(x), f1(x), f2(x), f3(x)] can further reduce this error. Therefore, it is necessary to use the correction boundary to further fine-tune the perspective correction. The specific steps are as follows: a. Calculate the four intersection points of the four lines f0(x), f1(x), f2(x), and f3(x) in the order of upper left -> upper right -> lower right -> lower left, recorded as intetsection_p0, intetsection_p1, intetsection_p2, intetsection_p3; b. Calculate the four points after perspective correction based on the above four intersection points, in the order of upper left -> upper right -> lower right -> lower left, recorded as pespective_p0, pespective_p1, pespective_p2, pespective_p3; c. According to the transformation of intetsection_pi->pespective_pi, i∈[0, 1, 2, 3], calculate the new perspective transformation matrix M2, and use the perspective transformation matrix M2 to fine-tune the perspective correction of Image2 to obtain Image3.

9. The method for correcting quadrilateral distortion based on close-range photography according to claim 8, characterized in that: The specific cutouts are: a. Since the ID images uploaded by taking photos generally have a unified target size requirement, it is necessary to scale the image Image3 to the target size; b. Select a standard card document image at the target size as the template image, then use the segment annotation tool to mark the four rounded corners in the image, and use the marked rounded corners as the rounded corner template and record them as mask; c. Use alpha image fusion technology to process the scaled Image3 and the mask template to complete the final cutout containing rounded corner information and place the cutout part on the specified background.

Images