Card information detection method, device, computer equipment and storage medium

Abstract

A card information detection method, device, computer equipment and storage medium, the method includes: obtaining a picture to be identified; performing text detection on the picture to be identified to obtain each text detection frame in the picture to be identified; determining the custom midpoint line of each text detection frame, and obtaining the attribute information of each text detection frame according to each custom midpoint line; the custom midpoint line is the connecting line of the midpoints of each short side of the text detection frame; clustering is performed using the attribute information of each text detection frame to obtain a clustering result; determining the card detection result according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction. The above method clusters the attribute information of the text detection frame to obtain a clustering result, and analyzing it can determine whether the picture contains more than two cards in a specific direction; determining the number and arrangement direction of the cards in the picture, this information can be used to identify the information in the card, and solve the problem of multiple card recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a card information detection method, device, computer equipment and storage medium. Background Art

[0002] With the development of technology, it is becoming more and more common to use automatic card detection instead of manual card detection. At present, the most commonly used technology for card recognition is OCR recognition technology. Using this technology to improve the automation level of the corresponding scenarios and reduce manual workload has always been a problem that needs to be solved.

[0003] In some application scenarios, the pictures uploaded by users may contain one or more overlapping cards, and the card information required in this application scenario is only partial information of one of the cards. In this case, it is necessary to accurately identify each card from the picture first, and then recognize the information in the card. The current general OCR model cannot solve the problem of accurate recognition of such pictures. Summary of the invention

[0004] Based on this, it is necessary to provide a card information detection method, device, computer equipment and storage medium that can identify multiple cards in order to solve the above technical problems.

[0005] A card information detection method, the method comprising:

[0006] Get the image to be identified;

[0007] Performing text detection on the image to be identified to obtain each text detection frame in the image to be identified;

[0008] Determine a custom midpoint line of each of the text detection frames, and obtain attribute information of each of the text detection frames according to each of the custom midpoint lines; the custom midpoint line is a connecting line of midpoints of each short side of the text detection frame;

[0009] Clustering is performed using the attribute information of each of the text detection boxes to obtain a clustering result;

[0010] The card detection result is determined according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0011] In one embodiment, the attribute information of the text detection frame includes: the horizontal coordinate of the center point of the text detection frame, and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system;

[0012] The clustering is performed using the attribute information of the text detection box to obtain a clustering result, including:

[0013] Clustering is performed twice using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions to obtain a first clustering result including one classification and a second clustering result including two classifications.

[0014] In one embodiment, determining the card detection result according to the clustering result includes:

[0015] Determine a first average distance between each point in the first clustering result and the center point of the clustering result, and determine a second average distance between each point in the second clustering result and the center point of the clustering result;

[0016] determining a ratio of the second distance average to the first distance average;

[0017] Comparing the ratio with a preset threshold value to obtain a comparison result;

[0018] The card detection result is determined according to the size comparison result.

[0019] In one embodiment, the card detection result is determined according to the comparison result between the first ratio and the second ratio and the corresponding preset thresholds, including:

[0020] If the size comparison result is that the ratio is greater than a preset threshold, it is determined that the card detection result includes the presence of more than two cards in the inclined direction or the horizontal direction.

[0021] In one embodiment, after determining the card detection result according to the clustering result, the method further includes:

[0022] According to the card detection result, the text detection frame is input into the text recognition module in a predetermined direction to obtain the text recognition result;

[0023] Template matching is performed on the text recognition result to obtain complete card information of a card.

[0024] In one embodiment, after performing template matching on the text recognition result to obtain complete card information of a card, the method further includes:

[0025] Obtain standard card information;

[0026] The target field information of the card information is filtered out from the complete card information in combination with the standard card information.

[0027] In one embodiment, after filtering out the target field information of the card information from the complete card information in combination with the standard card information, the method further includes:

[0028] Determine the applicant information and registration date of the marriage leave from the target field information;

[0029] Acquire corresponding marriage leave information from the system according to the applicant information;

[0030] The marriage leave information is matched with the applicant information and the registration date. If the information matches, the marriage leave is approved.

[0031] A card information detection device, the device comprising:

[0032] An acquisition module is used to acquire the image to be identified;

[0033] A text detection module, used to perform text detection on the image to be identified, and obtain each text detection frame in the image to be identified;

[0034] An attribute information acquisition module, used to determine a custom midpoint line of each of the text detection frames, and acquire attribute information of each of the text detection frames according to each of the custom midpoint lines; the custom midpoint line is a connecting line of the midpoints of the short sides of the text detection frames;

[0035] A clustering module, used to perform clustering using the attribute information of each of the text detection boxes to obtain a clustering result;

[0036] The card detection module is used to determine the card detection result according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the card information detection method when executing the computer program.

[0038] A computer-readable storage medium stores a computer program, which implements the steps of the card information detection method when executed by a processor.

[0039] The card information detection method, device, computer equipment and storage medium described above first perform text detection on the acquired image to be identified to obtain the information of the text detection frame, then determine the custom midpoint line of each text detection frame, and obtain the attribute information of each text detection frame based on each custom midpoint line, wherein the custom midpoint line is the connecting line of the midpoints of each short side of the text detection frame; clustering is performed using the attribute information of each text detection frame to obtain a clustering result; finally, based on the clustering result, a card detection result of whether there are more than two cards in a specific direction is obtained. The above method clusters the attribute information of the text detection frame identified from the image to obtain a clustering result, and analyzes the clustering result to obtain whether the image contains more than two cards in a specific direction; determines the number and arrangement direction of the cards in the image, and this information can be used to identify the information in the card to solve the problem of identifying multiple cards. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a process of detecting card information in an embodiment;

[0041] Figure 2 A schematic diagram of a process for determining a card detection result according to a clustering result in one embodiment;

[0042] Figure 3 A schematic diagram of a process of detecting card information in another embodiment;

[0043] Figure 4 A schematic diagram of a process of detecting card information in another embodiment;

[0044] Figure 5 A schematic diagram of a process of detecting card information in another embodiment;

[0045] Figure 6 It is a flowchart of a card information detection method in a specific embodiment;

[0046] FIG7 (1) is a schematic diagram of a specific embodiment in which a plurality of certificates are included in one picture;

[0047] FIG7 (2) is a schematic diagram showing a plurality of certificates in one picture in a specific embodiment;

[0048] Figure 8 It is a structural block diagram of a card information detection device in an embodiment;

[0049] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In one embodiment, Figure 1 As shown, a card information detection method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes steps S110 to S150.

[0052] Step S110, obtaining a picture to be identified.

[0053] The image to be identified is an image that needs to be identified by a card or certificate. The image to be identified can be uploaded by a user, or acquired by an image acquisition device, or acquired from a database. It can be understood that in other embodiments, the image to be identified that needs to be identified by a card or certificate can also be acquired by other means. The card or certificate refers to a card, certificate, etc.

[0054] Step S120, performing text detection on the image to be recognized to obtain each text detection frame in the image to be recognized.

[0055] Text refers to the expression form of written language. Any words fixed by writing can become a text. In this embodiment, text detection in the image to be identified refers to detecting all texts in the image to be identified, and obtaining text detection boxes corresponding to each text in the image to be identified. Text detection can be implemented by any method. The text is usually a quadrilateral. In a specific embodiment, based on the setting of conventional cards and certificates, it is generally a quadrilateral card or certificate. The algorithm for text detection in the image to be identified is EAST (An Efficient and Accurate SceneText Detector), which can realize effective detection of quadrilateral text.

[0056] A card usually contains multiple pieces of information, and each piece of information corresponds to a text detection box. Therefore, a card contains multiple text detection boxes. For example, an identity card usually contains information on one side, such as "name: **", "gender: *", "date of birth: **** / ** / **", "card number: ******", ... "validity period: **** / ** / ** to **** / ** / **", etc. Each piece of information can be used as a text, and corresponds to a text detection box during text detection.

[0057] Step S130 , determining the custom midpoint line of each text detection box, and obtaining attribute information of each text detection box according to each custom midpoint line.

[0058] The custom midpoint line is a connecting line of the midpoints of the short sides of the text detection box; in one embodiment, the text detection box is a rectangular box, the short side represents the side where the width of the rectangle is located, and a text detection box includes two short sides, and then the midpoints of the two short sides can be obtained, and the custom midpoint line of the text detection box can be obtained by connecting the two midpoints. It can be understood that one text detection box corresponds to one custom midpoint line.

[0059] After obtaining the text detection frames in the image to be recognized based on text recognition, for each text detection frame, the short sides thereof are respectively identified, and the midpoint position is determined in the short sides, and the midpoints of a text detection frame are connected to obtain a custom midpoint line corresponding to the text detection frame.

[0060] The attribute information of the text detection box refers to some attributes of the text detection box; in one embodiment, the attribute information of the text detection box includes the angle between the custom midpoint line corresponding to the text detection box and the horizontal axis of the preset coordinate system, and the coordinate value of the center point corresponding to the text detection box in the preset coordinate system. The preset coordinate system can select a suitable origin to set the coordinate axis according to actual needs.

[0061] Step S140 , clustering is performed using the attribute information of each text detection box to obtain a clustering result.

[0062] The process of dividing a collection of physical or abstract objects into multiple classes consisting of similar objects is called clustering. The cluster generated by clustering is a collection of data objects that are similar to objects in the same cluster and different from objects in other clusters. In this embodiment, the attribute information of the text detection box is used as the dimension of clustering, and clustering results corresponding to the attribute information can be obtained.

[0063] Among them, clustering can be achieved through any clustering method; in one embodiment, the method used to cluster the attribute information of the text detection box includes: K-Means (k-means clustering) clustering, mean shift clustering algorithm, single sliding window mean shift clustering, DBSCAN (Density-Based Spatial Clustering of Applications withNoise, density-based noise application spatial clustering) smiley face clustering or hierarchical clustering method.

[0064] Furthermore, in one embodiment, the attribute information of the text detection box includes: the horizontal coordinate of the center point of the text detection box in the preset coordinate system, and the angle between the custom midpoint line corresponding to the text detection box and the horizontal axis of the preset coordinate system. In this embodiment, clustering is performed using the attribute information of the text detection box to obtain a clustering result, including: taking the horizontal coordinate of the center point of the text detection box as the first dimension, taking the angle between the custom midpoint line corresponding to the text detection box and the horizontal axis of the preset coordinate system as the second dimension, clustering the text detection box twice based on the features of the first dimension and the second dimension, and obtaining a first clustering result including one classification, and a second clustering result including two classifications.

[0065] The center point of the text detection box is also the midpoint of the custom midpoint line. After determining the preset coordinate system and the position of the center point of the text detection box, the coordinate point of the center point in the preset coordinate system can be obtained. The horizontal coordinate of the center point is the coordinate value of the center point in the horizontal axis direction of the preset coordinate system; the angle between the custom midpoint line and the horizontal axis of the preset coordinate system represents the angle formed by the intersection of the custom midpoint line or the extension of the custom midpoint line with the horizontal axis of the coordinate system.

[0066] In one embodiment, the preset coordinate system is a two-dimensional rectangular coordinate system, and the preset coordinate system can be set according to actual conditions. For example, a corner point of the image to be identified can be used as the origin of the preset coordinate system, and the two mutually perpendicular directions selected can be used as the directions of the two coordinate axes of the preset coordinate system. In a specific embodiment, the upper left corner of the image to be identified is used as the origin of the coordinate system, the horizontal direction is used as the direction of the horizontal axis, and the vertical direction is used as the direction of the vertical axis to obtain the preset coordinate system. Among them, selecting different origins or different coordinate axis directions only affects the angle value between the custom midpoint line and the horizontal axis and / or the horizontal coordinate value of the center point of the text detection box, and has no effect on the step of clustering using attribute information.

[0067] Furthermore, in a specific embodiment, clustering can adopt K-means clustering method, that is, clustering is performed with k=1 and k=2 respectively based on the two clustering dimensions of the horizontal coordinate of the center point and the horizontal axis angle, and the obtained clustering results are recorded as the first clustering result and the second clustering result.

[0068] In a specific embodiment, K-Means clustering is performed using the attribute information of each text detection box; K-Means clustering is an iterative clustering analysis algorithm, the purpose of which is to divide the sample set into K clusters (i.e., categories) according to the distance (i.e., similarity) between samples for a given sample set; the steps are to divide the data into K groups, randomly select K objects as the initial cluster centers, and then calculate the distance between each object and each seed cluster center, and assign each object to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster. Each time a sample is assigned, the cluster center of the cluster will be recalculated based on the existing objects in the cluster. This process will be repeated until a certain termination condition is met. The termination condition can be that no (or a minimum number of) objects are reassigned to different clusters, no (or a minimum number of) cluster centers change again, and the error sum of squares is locally minimized.

[0069] In this embodiment, a preset coordinate system is used, and the horizontal coordinate of the center point of each text detection box and the angle between the custom midpoint line and the horizontal axis of the coordinate system are used as two clustering dimensions. Clustering is performed twice, and two clustering results of the text detection box can be obtained (including clustering results of one classification and two classifications); the clustering results reflect the characteristic information of the text detection box from two aspects: the horizontal coordinate of the center point and the angle between the custom midpoint line and the horizontal axis of the coordinate system, which can be used to analyze the information in the card later.

[0070] Step S150, determining a card detection result according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0071] After the clustering results are obtained, the card detection results can be obtained by analyzing them. Figure 2 As shown, determining the card detection result according to the clustering result includes steps S151 to S154.

[0072] Step S151, determining a first average distance between each point in the first clustering result and the center point of the clustering result; determining a second average distance between each point in the second clustering result and the center point of the clustering result.

[0073] Among them, the points included in the clustering results correspond to the text detection box; in this embodiment, the points in the clustering results correspond to two attributes: the horizontal coordinate of the center point of the text detection box, and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system.

[0074] It can be seen from the above embodiments that the first clustering result only contains one classification, that is, all points belong to the same cluster; in one embodiment, for the first clustering result, the average value of each point and the center point of the clustering result is determined, including: calculating the center point of the first clustering result, and calculating the distance between each point and the center point respectively, that is, calculating the distance average, which is recorded as the first distance average in this embodiment.

[0075] The second clustering result includes two classifications, that is, all points are divided into two different clusters, recorded as the first target cluster and the second target cluster; in one embodiment, the first center point and the second center point of the first target cluster and the second target cluster are determined, and the distance between each point in the first target cluster and the first center point is calculated respectively, and the distance between each point in the second target cluster and the second center point is calculated respectively, and then the average of all distance values ​​is taken, that is, the second distance average value.

[0076] In another embodiment, for the second clustering result, the average value of each point and the center point of the clustering result is determined, including: calculating the first center point of the first target cluster, and respectively calculating the distance between each point in the first target cluster and the first center point, so as to calculate the distance average value of the first target cluster; the distance average value of the second target cluster can be calculated in the same way, and then the average value of the distance average value of the first target cluster and the distance average value of the second target cluster in this embodiment is used as the second distance average value.

[0077] In one embodiment, the distance between two points can be calculated by Euclidean distance, cosine similarity, etc.

[0078] Step S152, determining the ratio of the second distance average value to the first distance average value.

[0079] Step S153, comparing the ratio with a preset threshold to obtain a comparison result.

[0080] The preset threshold can be set in advance according to the actual situation. In one embodiment, the determination of the preset threshold includes the steps of: randomly selecting n first group sample images containing only one card, and n second group sample images containing more than two cards in a specific direction; performing the following steps respectively with the two randomly selected groups of samples: text box detection, determining the custom midpoint line of the text box and the center point of the text detection box; performing two clusterings with the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis as two dimensions, obtaining a first sample clustering result containing one classification, and a second sample clustering result containing two classifications; then calculating the average distance between each point in each sample clustering result and the center point of the clustering result, respectively, and recording them as the first sample distance average and the second sample distance average. Then, for the sample group containing only one card, the ratio of the second sample distance average to the first sample distance average is calculated; for the sample group containing more than two cards, the ratio of the second sample distance average to the first sample distance average is calculated, and the clearly separated α is determined as the preset threshold according to the ratio of the two sample groups; in a specific embodiment, the preset threshold α=1.5.

[0081] Step S154, determining the card detection result according to the size comparison result.

[0082] Further, in one embodiment, determining the card detection result according to the size comparison result includes: if the size comparison result is a ratio greater than a preset threshold, determining that the card detection result includes the presence of more than two cards in an inclined direction or a horizontal direction.

[0083] In one embodiment, the specific direction refers to a horizontal direction or an inclined direction, wherein the inclined direction refers to a direction having a non-zero angle with the horizontal direction, wherein the horizontal direction is determined according to the image to be identified.

[0084] In this embodiment, clustering is performed twice using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal coordinate axis as two dimensions. Analysis is performed based on the two clustering results to determine whether there are two cards in the horizontal and inclined directions in the image to be identified. Determining the number of cards is helpful to improve the efficiency and accuracy of subsequent identification of card information.

[0085] The above-mentioned card information detection method, for the acquired image to be identified, first performs text detection to obtain the information of the text detection frame, then determines the custom midpoint line of each text detection frame, and obtains the attribute information of each text detection frame according to each custom midpoint line, wherein the custom midpoint line is the connecting line of the midpoints of each short side of the text detection frame; clustering is performed using the attribute information of each text detection frame to obtain a clustering result; finally, the card detection result of whether there are more than two cards in a specific direction is obtained according to the clustering result. The above-mentioned method clusters the attribute information of the text detection frame identified from the image to obtain a clustering result, and analyzes the clustering result to obtain whether the image contains more than two cards in a specific direction; determines the number and arrangement direction of the cards in the image, and this information can be used to identify the information in the card and solve the problem of multiple card identification.

[0086] Further, in one embodiment, if Figure 3 As shown, after the card detection result is determined according to the clustering result, step S310 and step S320 are also included.

[0087] Step S310: input the text detection frame into the text recognition module in a predetermined direction according to the card detection result to obtain the text recognition result.

[0088] In one embodiment, the predetermined direction can be preset according to actual needs. For example, in a specific embodiment, the predetermined direction is set to the vertical direction from top to bottom. Furthermore, if multiple text detection boxes are included in the same position in the vertical direction, each text detection box is input into the text recognition module for recognition in the order from left to right in the horizontal direction. In this embodiment, the vertical direction from top to bottom is used as the predetermined direction, which is more suitable for the situation where the order of text from top to bottom in the card is the same as the order of text from top to bottom in the picture to be recognized. In another embodiment, the direction corresponding to the order of text detection boxes from top to bottom in the card to be recognized can also be used as the predetermined direction.

[0089] In another embodiment, the predetermined direction may also be determined based on the text detection frame of the image to be recognized. After the text detection frame of the image to be recognized is detected, the long side and short side directions of the text detection frame may be obtained, and the short side direction of the text detection frame is determined as the predetermined direction. In this embodiment, considering that the direction of the card in the image to be recognized may be rotated in actual situations, determining the predetermined direction based on the text detection frame can improve the efficiency and accuracy of text recognition.

[0090] In one embodiment, a text detection frame is input into a text recognition module in a predetermined direction according to a card detection result to obtain a text recognition result, including: if the card detection result contains two cards in a specific direction, the text detection frames corresponding to the two clustering results are respectively input into the text recognition module in the predetermined directions to obtain text recognition results corresponding to the two clustering results; if the card result contains only one card in a specific direction, each text detection frame is input into the text recognition module in the predetermined direction.

[0091] In this embodiment, the text detection frame is input into the text recognition module in a predetermined direction according to the card detection result, and the obtained text recognition result is the text recognition result arranged in a certain order. And if it is determined according to the card detection result that the image to be recognized contains two cards, the two clustering results can be respectively subjected to text recognition, and the obtained text recognition results correspond to the text recognition results of the information in the two cards recognized in the image.

[0092] In one embodiment, text recognition of the text detection frame can be achieved in any manner; in a specific embodiment, a DenseNet (Densely Connected Convolutional Networks) in a convolutional neural network (CNN) is used, and the loss function is a weighted temporal connection classification Weight CTCLoss (Weight Connectionist temporal classisfication Loss).

[0093] Step S320: Perform template matching on the text recognition result to obtain complete card information of a card.

[0094] In one embodiment, template matching is performed on the text recognition results to obtain complete card information of a card, including: obtaining standard card information, matching the text recognition results with the key fields in the standard card information in sequence, and determining the text recognition results that are successfully matched with the key fields in a complete and continuous manner as the complete card information. The standard card information contains all the information that should be included in the card, which is recorded as key fields in this embodiment, such as "name", "date of birth", "ID number", etc.; matching each text recognition result with each key field of the standard card information in sequence, and determining the text recognition results that are successfully matched in a continuous manner as the complete card information. Among them, when each text recognition result is matched with each key field of the standard card information, the matching is performed in sequence from top to bottom in the standard card information.

[0095] In another embodiment, the text recognition results include two, and the two text recognition results are matched with the key fields of the standard card information in sequence, and the text recognition result with more consecutive successful matches is determined as the complete card information.

[0096] In this embodiment, text recognition is performed on each text detection frame in the image to be recognized, and the key fields of the standard card information are used to match the text recognition results in turn, so that the information of a single card can be obtained from the text recognition results corresponding to each text detection frame. Even if there are multiple cards in the image to be recognized, a single card containing complete information can be separated from them by the above method, which can be used for the analysis of card information in specific scenarios. At the same time, template matching using standard card information can also be used to identify the number and direction of cards in the case of more than two cards in the vertical direction, and separate a complete card from them.

[0097] In one embodiment, Figure 4 As shown, after performing template matching on the text recognition result to obtain complete card information of a card, it also includes step S410 and step S420:

[0098] Step S410, obtaining standard card information.

[0099] Step S420: Filter out target field information of the card information from the complete card information in combination with the standard card information.

[0100] In some embodiments, when the above card information detection method is applied to a specific scenario, the required information may be part of the information in a single card. The method in this embodiment can automatically filter out the required target field information from the identified card information.

[0101] In one embodiment, the target field can be determined according to the actual application scenario. For example, in the application scenario of identity authentication, the target field may be the "name", "ID number", etc. in the identity document; in the application scenario where a marriage certificate is used for process approval, the target field may be the "name of the certificate holder", "registration date", etc. in the marriage certificate, and the actual applicant is determined by the "name of the certificate holder".

[0102] Further, in one embodiment, if Figure 5 As shown, when the above method is applied to the application scenario of marriage leave audit, after the target field information of the card information is filtered out from the complete card information in combination with the standard card information, it also includes steps S510 to S530.

[0103] Step S510, determining the applicant information and registration date of the marriage leave from the target field information.

[0104] In this embodiment, the corresponding application information can be determined as marriage leave according to the image to be identified. In one embodiment, determining the applicant information of marriage leave from the target field information includes: determining the holder or name as the applicant information, and extracting the registration date from the text recognition result.

[0105] Step S520, obtaining corresponding marriage leave information from the system according to the applicant information.

[0106] In one embodiment, obtaining corresponding marriage leave information from the system means obtaining from the system whether the applicant has marriage leave and the available time of the marriage leave.

[0107] Step S530, matching the marriage leave information with the applicant information and the registration date, if the information matches, the marriage leave is approved.

[0108] In a specific embodiment, the identity of the applicant's information is verified, and the registration date is compared with the available time of the applicant's marriage leave to determine whether the marriage leave can be approved.

[0109] In this embodiment, the application for marriage leave can be automatically audited, which reduces the manual workload and improves the detection efficiency of marriage leave auditing.

[0110] In a specific embodiment, Figure 6 As shown, the card information detection method is applied to the scenario of automatic audit of marriage leave, including the steps of text detection, text recognition, template matching and verification output.

[0111] Among them, there are three types of overlapping situations of multiple cards: horizontal overlap: the two overlapping marriage certificates are almost parallel, and the overall angle is less than 10°; vertical overlap: the two overlapping marriage certificates are almost vertical, and the overall angle is less than 10°; inclined overlap: the two overlapping marriage certificates are neither parallel nor vertical, and the overall angle is greater than or equal to 10°.

[0112] The specific address, the card information detection method includes the following steps:

[0113] The first step is to perform text detection on the image to be identified and obtain the text detection frame (the text detection algorithm is EAST, which can realize the effective detection of quadrilateral text):

[0114] 1. Obtain the midpoint of the short side of each text detection box, connect the midpoints of the two short sides of each text detection box, and obtain a custom midpoint line.

[0115] 2. Calculate the center point of each text detection box (which can be determined by the midpoint of the custom midpoint line), and use the horizontal coordinate x of the midpoint as a dimension of clustering.

[0116] 3. Calculate the angle θ between the custom midpoint line in each detection frame and the coordinate axis x-axis, and use this angle θ as another dimension of clustering.

[0117] 4. Use the above two dimensional features as k-means clustering features for clustering.

[0118] 5. Set the number of cluster categories k=1 and k=2 respectively and perform clustering.

[0119] 6. Calculate the average values ​​d1 and d2 of the distance from each point to the center of the cluster to which it belongs when the number of cluster categories is 1 and 2 respectively.

[0120] 7. Compare the ratio between d1 and d2:

[0121] (1) If d2 / d1 > α, it is considered that there are two marriage certificate images in the oblique direction or the horizontal direction.

[0122] (2) If d2 / d1 ≤ α, it is considered that there is a marriage certificate image in the inclined direction or horizontal direction.

[0123] Among them, α is calculated by randomly selecting 10 single marriage certificate images and 10 images with two marriage certificates in an inclined direction or horizontal direction. The above 1-6 is performed on the two randomly selected groups of samples, and then d2 / d1 within the group is calculated to obtain a clearly separated α. In this embodiment, α=1.5. Then, according to the clustering results, the text detection boxes are transmitted from top to bottom to the recognition module.

[0124] The second step is to perform text recognition on the text detection box:

[0125] The text recognition module recognizes the sorted text boxes in each category in turn to obtain text recognition results.

[0126] In this embodiment, the recognition model used is DenseNet in the convolutional neural network (CNN), and the loss function is Weighted Temporal Connection Classification Weight CTC Loss.

[0127] The third step is to perform template matching on the text recognition results:

[0128] 1. In the standard card information, the key fields are in the following order from top to bottom: cardholder's name, registration date, name 1, ID number 1, name 2, ID number 2; the six fields are mainly three types of text information, name, date, and ID number. Therefore, the template is first set as: name, date, name, ID number, name, and ID number.

[0129] 2. Based on the text recognition results, the identified content in each category (category 1 or category 2, category 2 is the samples that the detection module determines to have overlapping) is classified as name, date, and ID number.

[0130] 3. For the text recognition results in each clustering result, a top-down template sliding window matching operation is performed. The window with the same clustering result and standard card information is recorded as a match, otherwise it is recorded as a mismatch.

[0131] 4. The matching result with the largest number of consecutive matching windows is selected as the final processing result.

[0132] Through the above operation, the marriage certificate information with more information in various situations (including vertical overlap) can be effectively extracted as the complete card information obtained by recognition. As shown in Figure 7 (1) and Figure 7 (2), respectively, there are multiple certificates in one picture in a specific embodiment.

[0133] Finally, verify the output:

[0134] After obtaining the complete card information, the verification process is performed, that is, the marriage leave application is audited. In one embodiment, if both the cardholder's name and name 1 exist, the following verification operations are performed:

[0135] 1. Compare the cardholder's name with name 1 to see if they are the same, and directly output the recognition result of name 1;

[0136] 2. If the cardholder's name is different from Name 1, and the cardholder's name recognition confidence is greater than the recognition confidence of Name 1, the cardholder's name recognition result shall be used as the recognition result of Name 1.

[0137] Finally, the validly recognized registration date, name 1, ID number 1, name 2, and ID number 2 are output. For the results that the target field cannot be recognized, a space is output to fill the space for manual verification.

[0138] The above card information detection method, for the case where multiple cards are included in the picture, separates a single card from it, and filters the target field information from the single card information; when applied to the application scenario of marriage leave audit, since the order of personal information corresponding to the target fields between the marriage certificates is consistent, the above method directly detects the text part of the target field personal information, and then completes the information mapping through the recognition results combined with post-processing, without the need to detect the target field keywords. By adding corresponding post-processing operations to the text detection module and the text recognition module respectively to solve the problem of multiple card folding, the separation of the target field can be automatically completed for the entire marriage leave application audit process, which can effectively improve the efficiency of marriage leave audit and save corresponding labor costs.

[0139] It should be understood that, although each step in each flow chart involved in the above-described embodiment is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each flow chart involved in the above-described embodiment may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0140] In one embodiment, Figure 8 As shown, a card information detection device is provided, including: an acquisition module 810, a text detection module 820, an attribute information acquisition module 830, a clustering module 840 and a card detection module 850, wherein:

[0141] The acquisition module 810 is used to acquire the image to be identified.

[0142] The text detection module 820 is used to perform text detection on the image to be recognized and obtain each text detection frame in the image to be recognized.

[0143] The attribute information acquisition module 830 is used to determine the custom midpoint line of each text detection box and acquire the attribute information of each text detection box according to each custom midpoint line; the custom midpoint line is a connecting line of the midpoints of each short side of the text detection box.

[0144] The clustering module 840 is used to perform clustering using the attribute information of each text detection box to obtain a clustering result.

[0145] The card detection module 850 is used to determine a card detection result according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0146] The above-mentioned card information detection device, for the acquired image to be identified, first performs text detection to obtain the information of the text detection frame, then determines the custom midpoint line of each text detection frame, and obtains the attribute information of each text detection frame according to each custom midpoint line, wherein the custom midpoint line is the connecting line of the midpoints of each short side of the text detection frame; clustering is performed using the attribute information of each text detection frame to obtain a clustering result; finally, a card detection result of whether there are more than two cards in a specific direction is obtained according to the clustering result. The above-mentioned device clusters the attribute information of the text detection frame identified from the image to obtain a clustering result, and analyzes the clustering result to obtain whether the image contains more than two cards in a specific direction; determines the number and arrangement direction of the cards in the image, and this information can be used to identify the information in the card to solve the problem of multiple card identification.

[0147] In one embodiment, the attribute information of the text detection box includes: the horizontal coordinate of the center point of the text detection box, and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system; in this embodiment, the above-mentioned clustering module 840 is specifically used to: perform two clusterings using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions, respectively, to obtain a first clustering result including one classification, and a second clustering result including two classifications.

[0148] In one embodiment, the card detection module 850 of the above-mentioned device includes: a distance average value determination unit, which is used to determine the first distance average value between each point in the first clustering result and the center point of the clustering result, and determine the second distance average value between each point in the second clustering result and the center point of the clustering result; a ratio determination unit, which is used to determine the ratio of the second distance average value to the first distance average value; a size comparison unit, which is used to compare the ratio with a preset threshold value to obtain a size comparison result; and a card detection result determination unit, which is used to determine the card detection result according to the size comparison result.

[0149] In one embodiment, the size comparison unit of the above device is specifically used to: if the size comparison result is that the ratio is greater than a preset threshold, determine that the card detection result includes that there are more than two cards in the inclined direction or the horizontal direction.

[0150] In one embodiment, the above-mentioned device also includes: a text recognition module, which is used to input the text detection box into the text recognition module in a predetermined direction according to the card detection result to obtain the text recognition result; a single card information output module is used to perform template matching on the text recognition result to obtain the complete card information of a card.

[0151] In one embodiment, the above device further includes: a standard card information acquisition module, used to acquire standard card information; and a target field output module, used to filter out target field information of the card information from the complete card information in combination with the standard card information.

[0152] In one embodiment, the above-mentioned device also includes: an applicant information determination module, which is used to determine the applicant information and registration date of the marriage leave from the target field information; a marriage leave information acquisition module, which is used to obtain the corresponding marriage leave information from the system according to the applicant information; and a marriage leave audit module, which is used to match the marriage leave information with the applicant information and registration date. If the information matches, the marriage leave is approved.

[0153] The specific definition of the card information detection device can be found in the definition of the card information detection method above, which will not be repeated here. Each module in the above card information detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0154] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a card information detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0155] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0157] Obtain a picture to be identified; perform text detection on the picture to be identified to obtain a text detection frame in the picture to be identified; determine a custom midpoint line for each text detection frame, and obtain attribute information of each text detection frame based on each custom midpoint line; the custom midpoint line is a connecting line of the midpoints of each short side of the text detection frame; perform clustering using the attribute information of each text detection frame to obtain a clustering result; determine a card detection result based on the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0158] In one embodiment, when the processor executes the computer program, it also implements the following steps: clustering twice using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions, to obtain a first clustering result including one classification and a second clustering result including two classifications.

[0159] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining a first distance average value between each point in the first clustering result and the center point of the clustering result, and determining a second distance average value between each point in the second clustering result and the center point of the clustering result; determining a ratio of the second distance average value to the first distance average value; comparing the ratio with a preset threshold value to obtain a size comparison result; and determining a card detection result based on the size comparison result.

[0160] In one embodiment, when the processor executes the computer program, the following steps are further implemented: if the size comparison result is that the ratio is greater than a preset threshold, determining that the card detection result includes the presence of more than two cards in the inclined direction or the horizontal direction.

[0161] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the card detection result, the text detection frame is input into the text recognition module in a predetermined direction to obtain the text recognition result; and the text recognition result is template matched to obtain the complete card information of a card.

[0162] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining standard card information; and filtering out target field information of the card information from the complete card information in combination with the standard card information.

[0163] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining the applicant information and registration date of the marriage leave from the target field information; obtaining the corresponding marriage leave information from the system according to the applicant information; matching the marriage leave information with the applicant information and registration date, and if the information matches, the marriage leave is approved.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining a picture to be identified; performing text detection on the picture to be identified to obtain a text detection frame in the picture to be identified; determining a custom midpoint line of each text detection frame, and obtaining attribute information of each text detection frame according to each custom midpoint line; the custom midpoint line is a connecting line of the midpoints of each short side of the text detection frame; clustering is performed using the attribute information of each text detection frame to obtain a clustering result; determining a card detection result according to the clustering result; the card detection result includes whether there are more than two cards in a specific direction.

[0165] In one embodiment, when the processor executes the computer program, it also implements the following steps: clustering twice using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions, to obtain a first clustering result including one classification and a second clustering result including two classifications.

[0166] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining a first distance average value between each point in the first clustering result and the center point of the clustering result, and determining a second distance average value between each point in the second clustering result and the center point of the clustering result; determining a ratio of the second distance average value to the first distance average value; comparing the ratio with a preset threshold value to obtain a size comparison result; and determining a card detection result based on the size comparison result.

[0167] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: if the size comparison result is that the ratio is greater than a preset threshold, determining that the card detection result includes the presence of more than two cards in the inclined direction or the horizontal direction.

[0168] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the card detection result, the text detection frame is input into the text recognition module in a predetermined direction to obtain the text recognition result; and the text recognition result is template matched to obtain the complete card information of a card.

[0169] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining standard card information; and filtering out target field information of the card information from the complete card information in combination with the standard card information.

[0170] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the applicant information and registration date of the marriage leave from the target field information; obtaining the corresponding marriage leave information from the system according to the applicant information; matching the marriage leave information with the applicant information and the registration date, and if the information matches, the marriage leave is approved.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A card information detection method, characterized in that: The method comprises: Get the image to be identified; Performing text detection on the image to be identified to obtain a text detection frame in the image to be identified; Determining a custom midpoint line for each text detection frame, and obtaining attribute information of each text detection frame based on each custom midpoint line; the custom midpoint line is a line connecting the midpoints of each short side of the text detection frame, and the attribute information of the text detection frame includes the horizontal coordinate of the center point of the text detection frame and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system; Performing clustering twice using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions, respectively, to obtain a first clustering result containing one category and a second clustering result containing two categories; Determine a first average distance between each point in the first clustering result and the center point of the clustering result, and determine a second average distance between each point in the second clustering result and the center point of the clustering result; determining a ratio of the second distance average to the first distance average; Comparing the ratio with a preset threshold to obtain a comparison result; If the size comparison result is that the ratio is greater than a preset threshold, it is determined that the card detection result includes the presence of more than two cards in the oblique direction or the horizontal direction.

2. The method according to claim 1, characterized in that After determining the card detection results based on the clustering results, the following steps are also included: According to the card detection result, the text detection frame is input into the text recognition module in a predetermined direction to obtain the text recognition result; Perform template matching on the text recognition result to obtain complete card information of a card.

3. The method according to claim 2, characterized in that After performing template matching on the text recognition result to obtain complete card information of a card, the following steps are also included: Obtain standard card information; The target field information of the card information is filtered out from the complete card information in combination with the standard card information.

4. The method according to claim 3, characterized in that After filtering out target field information of the card information from the complete card information in combination with the standard card information, the method further includes: Determining the applicant information and registration date of the marriage leave from the target field information; Obtain corresponding marriage leave information from the system according to the applicant information; The marriage leave information is matched with the applicant information and the registration date. If the information matches, the marriage leave is approved.

5. A card information detection device, characterized in that: The device comprises: The acquisition module is used to obtain the image to be identified; A text detection module is used to perform text detection on the image to be identified to obtain each text detection frame in the image to be identified; an attribute information acquisition module, configured to determine a custom midpoint line for each of the text detection frames, and acquire attribute information of each of the text detection frames based on each of the custom midpoint lines; the custom midpoint line being a line connecting the midpoints of the short sides of the text detection frame, and the attribute information of the text detection frame including the horizontal coordinate of the center point of the text detection frame and the angle between the custom midpoint line and the horizontal axis of a preset coordinate system; A clustering module, configured to perform two clustering operations using the horizontal coordinate of the center point of the text detection box and the angle between the custom midpoint line and the horizontal axis of the preset coordinate system as two clustering dimensions, respectively, to obtain a first clustering result including one classification and a second clustering result including two classifications; The card detection module is configured to determine an average first distance between each point in the first clustering result and the center point of the clustering result, and an average second distance between each point in the second clustering result and the center point of the clustering result; determine a ratio of the second distance average to the first distance average; compare the ratio with a preset threshold to obtain a size comparison result; and if the size comparison result indicates that the ratio is greater than the preset threshold, determine that the card detection result includes the presence of more than two cards in an oblique or horizontal direction.

6. The device according to claim 5, characterized in that It also includes a text recognition module, which is used to: input the text detection frame into the text recognition module in a predetermined direction according to the card detection result to obtain a text recognition result; and perform template matching on the text recognition result to obtain complete card information of a card.

7. The device according to claim 6, characterized in that It also includes a standard card information acquisition module, which is used to: acquire standard card information; and filter out target field information of the card information from the complete card information in combination with the standard card information.

8. The device according to claim 7, characterized in that It also includes an applicant information determination module, which is used to: determine the applicant information and registration date of the marriage leave from the target field information; obtain corresponding marriage leave information from the system based on the applicant information; match the marriage leave information with the applicant information and the registration date, and if the information matches, the marriage leave is approved.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images