Card information identification method, device and terminal

Abstract

The present application discloses a card information recognition method, device and terminal, the method comprising: obtaining a card image to be recognized; detecting at least part of the attribute identifiers in the card image; obtaining the recognition direction of the card image to be recognized according to the attribute identifiers; recognizing the card image according to the recognition direction to obtain recognition information. Since the recognition direction can be obtained, the card image at any angle can be recognized, which can reduce the processing process of generating image angle guidance information during the recognition process, thereby simplifying the recognition process and improving the recognition efficiency.

Description

Technical Field

[0001] The present application relates to the technical field, and in particular to a card information recognition method, device and terminal. Background Art

[0002] With the development of Internet technology, in business areas such as finance, transportation, and security inspection, the automatic extraction of card information such as bank cards or identity documents is required, which can replace a large amount of tedious manual entry.

[0003] At present, in the existing card information recognition, in order to accurately extract relevant information, it is necessary to generate guidance information each time to guide users to shoot cards at a standard angle, or require users to upload card pictures in a standard format, and then perform real-time recognition on the obtained card pictures. It can be seen that the existing card information recognition method needs to generate guidance information each time, which makes the card information recognition process cumbersome and inefficient. Summary of the invention

[0004] In view of this, the present application provides a card information recognition device and terminal, so that card images at any angle can be recognized, thereby improving the processing efficiency of the recognition process and simplifying the recognition process.

[0005] To achieve the above objectives, on the one hand, the present application provides a card information identification method, comprising:

[0006] Obtain the card image to be identified;

[0007] Detecting at least part of the attribute identifier in the card image;

[0008] According to the attribute identifier, obtaining a recognition direction of the card image to be recognized;

[0009] The card image is identified according to the identification direction to obtain identification information.

[0010] In a possible implementation, obtaining the card image to be identified includes:

[0011] Obtain the original image of the card to be identified;

[0012] Performing feature area recognition on the original image of the card to be identified to obtain the central area of ​​the card to be identified;

[0013] Generate an image acquisition area of ​​the original image according to the central area, wherein the image acquisition area includes the attribute identifier;

[0014] The image acquisition area of ​​the original image is subjected to image acquisition to obtain a card image to be identified.

[0015] In yet another possible implementation, the detecting at least part of the attribute identifier in the card image includes:

[0016] The card image is input into a field detection model to obtain attribute identifiers of a field area and a target field area of ​​the card image, wherein the target field area represents a field area having a preset field format, and the attribute identifier represents a front field area of ​​the target field area.

[0017] In another aspect, the present application further provides a card information recognition device, comprising:

[0018] An image acquisition unit, used to acquire the card image to be identified;

[0019] An identification detection unit, used to detect at least part of the attribute identification in the card image;

[0020] A direction acquisition unit, used to obtain the recognition direction of the card image to be recognized according to the attribute identifier;

[0021] The information recognition unit is used to recognize the card image according to the recognition direction to obtain recognition information.

[0022] On the other hand, the present application also provides a terminal, including:

[0023] Processor and memory;

[0024] Wherein, the memory is used to store programs;

[0025] The processor is used to execute a program stored in the memory, wherein the program is at least used to:

[0026] Obtain the card image to be identified;

[0027] Detecting at least part of the attribute identifier in the card image;

[0028] According to the attribute identifier, obtaining the recognition direction of the card image to be recognized;

[0029] The card image is identified according to the identification direction to obtain identification information.

[0030] It can be seen that when identifying the card information, it is only necessary to obtain the card image to be identified and detect at least part of the attribute identifiers in the card image, and obtain the identification direction from the attribute identifiers. In this way, the card image can be identified according to the identification direction to obtain the identification information. Since the identification direction can be obtained, the card image at any angle can be identified, which can reduce the processing process of generating image angle guidance information during the identification process, thereby simplifying the identification process and improving the recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0032] Figure 1 A schematic diagram showing the composition and architecture of a card information recognition system according to an embodiment of the present application is shown;

[0033] Figure 2 A schematic diagram of a process interaction of a card information identification method according to an embodiment of the present application is shown;

[0034] Figure 3 A schematic diagram showing a process of a card image acquisition method according to an embodiment of the present application is shown;

[0035] Figure 4 A schematic diagram of implementing card image detection using a card detection model according to an embodiment of the present application is shown;

[0036] Figure 5 A schematic diagram showing a card image detection area in an embodiment of the present application;

[0037] Figure 6 A schematic diagram showing a method for obtaining an attribute identifier according to an embodiment of the present application is shown;

[0038] Figure 7 A schematic diagram of labeling training samples of a field detection model according to an embodiment of the present application is shown;

[0039] Figure 8 A schematic diagram showing a flow chart of a method for obtaining identification information according to an embodiment of the present application;

[0040] Fig. 9 A schematic diagram of a scenario for obtaining identification information according to an embodiment of the present application is shown;

[0041] Fig.10 A schematic diagram showing the composition of a card information recognition network according to an embodiment of the present application is shown;

[0042] Fig.11 A schematic diagram of identifying identity card information in an embodiment of the present application is shown;

[0043] Fig.12 A schematic diagram showing the structure of a card information recognition device according to an embodiment of the present application is shown;

[0044] Fig.13A schematic diagram of the structure of a terminal according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The card certificate in the scheme of the present application represents a type of certificate with the same shape and format, such as the more common resident identity card, driver's license, bank card or other similar membership cards, which are used to record various materials or represent the identity information of the user in a certain scenario. And the card certificate image has similar characteristics, including: the document structure of the card certificate image is fixed, such as the relative position of each field of the identity card is roughly fixed, and the bank card includes a bank card number field, etc.; the shooting scene of the card certificate image is diversified, such as the shooting terminal, shooting environment, and shooting angle will vary, which will make the angle of the card certificate image finally obtained also diversified. Among them, in the embodiment of the present application, the recognition of the card certificate information refers to the recognition of the text and other related information recorded on the appearance of the card certificate, which may include the recognition of all text information, and may also include the recognition of the specified field information. For example, the text information on the resident identity card is recognized, and the recognition result obtained may include relevant information such as the identity card number, name, gender and the validity period of the certificate. Correspondingly, if the recognition field only includes the name and identity card number fields, the recognition result only includes the text information corresponding to the name and identity card number fields.

[0046] In order to facilitate understanding of the card information identification method of the present application, the following introduces the system to which the card information identification method of the present application is applicable. Figure 1 , which shows a schematic diagram of the composition architecture of a card information recognition system of the present application.

[0047] like Figure 1 As shown, the card information recognition system provided by the embodiment of the present application includes: a terminal 10 and a server 20. The terminal 10 and the server 20 are connected to each other through a network 30.

[0048] The terminal 10 may be a mobile terminal such as a mobile phone or a tablet computer, or a fixed terminal such as a personal computer with an image acquisition component, such as a computer connected to a camera.

[0049] In the embodiment of the present application, the terminal 10 can collect the card image that needs to be identified through the image acquisition components such as the camera set or connected thereto, and transmit the collected card image to the server 20 through the network 30, so that the server 20 obtains the card image to be identified; the terminal 10 can also send the acquired card image to the server 20 in other ways, for example, the terminal sends its pre-stored card image to the server.

[0050] Correspondingly, the server 20 will identify the information in the card certificate image sent by the terminal 10, for example, identify the string contained in the card certificate image, and match the identification information of the field and field content corresponding to the identified string. After the server 20 obtains the identification result, the identification result can be returned to the terminal 10 through the network 30, so that the terminal 10 can output the identification information corresponding to the identification result. It should be noted that after the server obtains the identification information, the identification information can be recorded, or the identification information can be sent to a designated destination terminal. For example, when the card certificate information identification method is applied to a bank counter business processing scenario, the card certificate is a bank card. If the terminal is a terminal for bank customers, the terminal collects the bank card image of the current bank customer and sends the bank card image to the server, so that the server can obtain the identification information based on the bank card image. Since the bank customer needs to perform corresponding business based on the current bank card, the server will send the identification information to the terminal of the teller who is currently processing the business, so that the current teller can process the business request of the current bank customer based on the identification information.

[0051] In one possible scenario, in order to reduce the traffic consumed by the terminal to transmit images, the terminal 10 can also extract part of the image from the image as the target image to be transmitted to the server after acquiring the card image to be identified, wherein the target image is part of the image acquired by the terminal. Of course, in some application fields, in order to reduce the hardware cost of the terminal 10, the terminal 10 only has the function of image acquisition, and the terminal 10 will directly send the acquired image to the server 20. In order to improve the processing performance of information recognition of the server 20, the server 20 can perform preliminary processing on the acquired image, first obtain the target image, and then perform corresponding character recognition.

[0052] Optionally, the terminal may run an application, where the application is used to establish a communication connection with the server, and the terminal exchanges information with the server through the application.

[0053] The following is a detailed description of the interaction process between the terminal and the server.

[0054] For example, see Figure 2 , which shows a schematic diagram of the process interaction of an embodiment of a card information identification method of the present application. The method of this embodiment may include:

[0055] S201. The terminal collects the card image to be detected.

[0056] S202: The terminal sends the card image to the server.

[0057] When the terminal collects the card image, it can collect it based on the image collection instruction. The image collection instruction is used to trigger the terminal to collect the image of the card to be identified. That is, the terminal uses the terminal's camera to collect the image of the card to be identified through the image collection instruction. For example, the terminal scans or photographs the card to obtain the card image.

[0058] In one possible scenario, the terminal is connected to the server, and after the server generates an image acquisition instruction and sends the image acquisition instruction to the terminal, the terminal performs image acquisition of the currently existing card. For example, when registering identity information, the server interacts with the first terminal used by the staff, and when the staff sends an image acquisition instruction to the server through the first terminal, the server forwards the image acquisition instruction to the second terminal for image acquisition, triggering the image acquisition instruction of the second terminal to perform image acquisition of the ID card.

[0059] In another possible scenario, after the card information recognition function is started in the terminal application, the application will call the terminal camera to collect the image information of the card to be recognized. At this time, the image acquisition instruction can be an instruction generated by starting the application, or the user can select or input corresponding instruction information after starting the application as the image acquisition instruction.

[0060] When the terminal collects the card image, the card positioning model can also be called, that is, the position of the card image is determined by the card positioning model, so that the card image with a precise position can be obtained in the captured or collected image. Specifically, the positioning process of the card image by the card positioning model and the training process of the card positioning model will be described in detail in the following embodiments.

[0061] S203: The server detects at least part of the attribute identifiers in the card image.

[0062] The attribute identifier may represent the unique attribute information in the card image, or the region of the unique attribute information; it may also represent part of the unique attribute information, or the sub-region corresponding to the unique attribute information region. For example, when the card to be identified is an ID card, its attribute identifier may be the ID card number, or the detection region where the ID card number is located, or the first half of the ID card number.

[0063] For card types with the same format, the attribute identifier can be obtained based on the structural characteristics and information layout of the card. For example, for bank cards of the same type and with a bank card chip from the same bank, the position of the attribute identifier can be determined based on the correspondence between the attribute identifier and the position of the chip in the bank card. The position of the attribute identifier can also be determined based on the length of the string of the attribute identifier. For example, in an identity card, the attribute identifier is the identity card number, which is the longest string of text information in the identity card. Therefore, the position of the longest string in the identity card can be identified, and this position is the position of the attribute identifier.

[0064] However, if the above method is used to determine the attribute identification of the card image, it is necessary to distinguish different types of cards, or there will be different subtypes under a certain type of card. This kind of recognition of feature positions will make the recognition process slower, and due to the use of different recognition templates for recognition, the process will also become cumbersome and the accuracy rate will be relatively low.

[0065] Therefore, in another possible implementation, the attribute identification is detected by a field detection model, that is, the field detection model is used to detect the position of each field in the card image, and the target field can be detected at the same time, the target field represents the field where the unique identification information in the card image is located, and the front part of the target field can also be detected. In order to facilitate the subsequent determination of the recognition direction, the front part of the target field is preferably used as the attribute identification. Specifically, the detection process of the attribute identification and the field detection model will be described in detail in the following embodiments.

[0066] S204: The server obtains the recognition direction of the card image to be recognized according to the attribute identifier.

[0067] Since the attribute identifier is part of the target field area of ​​the card image, mask processing can be performed on the target field area and the area corresponding to the attribute identifier, so as to determine the four vertex coordinates and the order of coordinate points of the target area according to the predicted value of the mask image of the attribute identifier area and the predicted value of the mask image of the area corresponding to the non-attribute identifier, and then determine the recognition direction of the current target field. Since the layout of the card fields all adopts the same writing order, after determining the recognition direction of the target field, the recognition direction of the card image and other target fields is obtained.

[0068] The above-mentioned determination of the recognition direction of the card image is universal. As long as the attribute identifier and the field where the attribute identifier is located are obtained, the recognition direction can be obtained according to the mask image processing. It is applicable to different types of card images.

[0069] Of course, for cards with special formats and fixed formats with strong universality, the recognition direction can also be determined by the correspondence between the attribute identifier and the position pair of specific features. It should be noted that if this technical means is used, the type and format of the card must be determined first before subsequent direction recognition can be performed. For example, a citizen's ID card is a type of card with a unified format feature. After identifying its attribute identifier, that is, the ID card number, the recognition direction can be determined by the position correspondence between the ID card number and the ID card photo.

[0070] Since the attribute identifier of the card is a unique identifier that can reflect the card information, its structure is usually arranged in a specific format. In another possible implementation of the present application, an adjacent string of the attribute identifier can be extracted according to the encoding structure of the attribute identifier, and then the recognition direction of the card image can be determined according to the character sequence of the string and the fixed encoding sequence.

[0071] S205: The server identifies the card image according to the identification direction to obtain identification information.

[0072] S206, the server sends the identification information to the terminal;

[0073] S207. The terminal outputs identification information.

[0074] After obtaining the recognition direction, the direction of the acquired card image can be adjusted according to the recognition direction. For example, the card image can be flipped at a corresponding angle according to the recognition direction so that the text presented by the flipped card image is in a direction that can be recognized by the text recognition model. Alternatively, the recognition direction of the text collection frame can be adjusted so that the recognition direction of the text collection frame matches the recognition direction of the card image, so that the corresponding text information can be collected.

[0075] When collecting text information, character information such as text can be extracted according to an OCR (Optical Character Recognition) algorithm, or recognition information can be extracted according to a character recognition model.

[0076] It should be noted that when identifying character information, all character information may be identified, or characters of field contents corresponding to certain predetermined identification fields may be identified to obtain identification information.

[0077] It can be seen that when the card to be identified is identified, the identification direction of the card image to be identified can be obtained according to the attribute identifier of the card image, and then the card image is identified according to the identification direction to obtain identification information. In this way, no matter how the card is placed, and based on the card image collected at the current placement angle, correct character recognition can be performed according to the obtained identification direction, so that the card taken at any angle can be accurately processed, making the identification process simple, the recognition efficiency high, and also improving the recognition accuracy.

[0078] Correspondingly, after the server obtains the identification information, it can send the identification information to the terminal so that the terminal can output the identification information, or it can send the identification information to other output terminals.

[0079] In a possible implementation of the embodiment of the present application, the card image can be captured or the captured card image can be preprocessed according to the image capture area generated according to a predetermined rule. Figure 3 , which shows a schematic diagram of a process of obtaining a card image, the method comprising:

[0080] S301, obtaining the original image of the card to be identified;

[0081] S302, performing feature area recognition on the original image of the card to be identified to obtain the central area of ​​the card to be identified;

[0082] S303, generating an image acquisition area of ​​the original image according to the central area, wherein the image acquisition area includes an attribute identifier;

[0083] S304: perform image acquisition on the image acquisition area of ​​the original image to obtain a card image to be identified.

[0084] In this embodiment, the method can be applied on a terminal or on a server. If applied on a terminal, it corresponds to the process of the terminal collecting the card image, and if applied on a server, it corresponds to the process of the server processing the card image uploaded by the terminal.

[0085] Among them, when the original image of the card to be identified is subjected to feature area identification, the feature area can represent the area divided according to the features of different pixels of the original image of the card, and can represent the area with a specific position relationship with a specific attribute field. Then, after identifying each feature area, the central area of ​​the card to be identified can be obtained, and the central area can represent the area at the center of the card image, or the area where the fields are concentrated in the card image.

[0086] After the central area is obtained, the image acquisition area of ​​the original image can be generated according to the central area. Specifically, the central area can be expanded to obtain the image acquisition area, which includes the attribute identifier. Then, image acquisition is performed according to the image acquisition area to obtain the card image to be identified. In this way, a more accurate card image to be identified can be obtained, which can solve the problem in the prior art that the card image detection template based on the feature cannot obtain the card image of the partially missing card.

[0087] For example, when information recognition is performed based on the template matching scheme in the prior art, the position information of the card is usually obtained through an edge detection algorithm. If the edges of some card are blocked or incomplete, the card image cannot be obtained according to the edge detection algorithm in the prior art. In the embodiment of the present application, an image acquisition area can be generated according to a determined central area, and a card image can be detected, so that even if the card is incomplete, the position of the card can be located, thereby obtaining a card image.

[0088] In order to accurately identify the characters in the card, and to improve the efficiency of information recognition by the server. In an embodiment of the present application, a card detection model is provided, which is obtained by training a fully convolutional network (FCN). Correspondingly, the training method of the card detection model is as follows: obtain multiple card images, annotate the regional coordinates and attribute categories of each card image according to the pixel characteristics of different regions, obtain annotated data, and use multiple card images including firecracker data as a training set. Input the images in the training set into the FCN, and the FCN outputs the detected regional coordinates and regional attribute categories. According to the difference between the output regional coordinates and attribute categories and the annotated data, the parameters of the FCN are adjusted, and finally a trained FCN model, i.e., a card detection model, is obtained.

[0089] See also Figure 4 , which shows a schematic diagram of using a card detection model to implement card image detection in an embodiment of the present application, and in this embodiment, the card image detection of a bank card is taken as an example. Figure 4In the embodiment, the bank card image 401 to be detected is input into the card detection model 402, wherein the card detection model 402 is obtained by training the fully convolutional neural network, and the bank card image 401 is image information including interference information, wherein the interference information may include background information or certain occlusion information. After the card detection model 402 obtains the bank card image 401, it will detect the image and output an image detection result 403 matching the bank card image, wherein the image detection result 403 includes a background area image 4031, a center area image 4032, and an edge area image 4033. That is, the card detection model is used to judge each pixel in the input bank card image, and predict whether the pixel belongs to the background area, the center area, or the edge area.

[0090] Correspondingly, when training the card detection model, the true value mask can also be obtained according to the known image area position to optimize the model. For example, the category of the background area of ​​the true value mask is set to 0, the value of the central area of ​​the card is set to 1, and the value of the edge area of ​​the card is set to 2. The loss function (such as cross-entropy loss) can be used to optimize the full convolutional neural network in the card detection model. The loss function is used to represent the gap between the predicted value and the answer. When training the neural network, the loss function is continuously reduced by continuously changing all the parameters in the neural network, so as to train a more accurate neural network model. The true value mask can be obtained according to the known image area position to obtain a mask image, wherein the mask image is a specific image used for masking the image to be processed, and can also be regarded as a template. Specifically, in digital image processing, the mask image can be a two-dimensional matrix array, or a multi-value image. Specifically, the server can generate a corresponding mask image according to the central area and the non-central area, which is used to segment the central area of ​​the card image to be identified to obtain the card image. In a possible implementation, the non-central area can be marked as 0, and the central area can be marked as 1, and a mask image including the marking values ​​of the non-central area and the central area can be obtained, that is, a two-dimensional matrix array composed of 0 and 1. During image mask processing, the card image to be identified is blocked with a selected image, graphic or object to control the image processing area or processing process. Specifically, the card image to be identified can be masked according to the mask image, the non-central area can be blocked, and a central area image including pixels in the central area can be obtained.

[0091] In another implementation, for a card image with interference information, it is also possible to determine the pixels of the card image to predict whether each pixel in the current card image belongs to a certain area of ​​the card.

[0092] For example, based on the category information of the card image, the pixel value information of the pixel points of each category of the card is calculated. When the pixel value of the pixel points in a certain area of ​​the card is identified to match the reference pixel value of the area, the area is determined as an area with matching reference pixel values, thereby identifying various areas of the card.

[0093] After the card is detected and each area of ​​the card is obtained, the image detection area can be determined based on the identified area. When determining the image detection area, it can be determined based on the field information to be identified. Usually, the key information of the card is generally concentrated in the central area of ​​the card, so the image detection area will be determined based on the detected central area, so that the edge area can be filtered out, and the recognition efficiency will be higher in the subsequent character recognition process.

[0094] For example, see Figure 5 , which shows a schematic diagram of a card image detection area. By taking a connected domain from the mask of the central area of ​​the card, a connected domain 501 is obtained, and then a circumscribed quadrilateral 502 of the central area of ​​the card is obtained. By expanding the quadrilateral, an accurate card image detection frame 503 can be obtained.

[0095] After the card detection frame 503 is obtained, the collected original image of the card to be identified is detected using the card detection frame 503 to obtain a target image, which is the card image to be identified.

[0096] It should be noted that the card detection model can also process images of multiple cards with overlapping areas. For example, when identifying the card information of several cards, the cards are usually stacked for quick identification, and are identified one by one from top to bottom. It is possible that part of the information of the next card is exposed under the previous card, so that a field in the captured card image may be a repeated field from different cards. In view of this scenario, an implementation method is provided in the embodiment of the present application:

[0097] Performing feature region recognition on the original image of the card to be recognized to obtain a plurality of feature sub-regions;

[0098] If a group of target characteristic sub-regions matching the card to be identified are screened out from the plurality of characteristic sub-regions, a central region of the card to be identified is acquired from the target sub-characteristic sub-regions.

[0099] After obtaining several feature sub-regions through the card detection model, it is necessary to screen out a group of target feature sub-regions that match the card to be identified. The target feature sub-regions include background regions, edge regions and central regions, and each region has adjacent or connected dividing lines or coordinate points. This ensures that the central region is the central region of the card to be identified. The image acquisition area subsequently expanded based on the central region can capture the relevant fields of the current card to be identified, avoiding the influence of other stacked card images.

[0100] See also Figure 6 , which shows a schematic diagram of a method for obtaining attribute identification in an embodiment of the present application. In this embodiment, the attribute identification is obtained through a field detection model. Taking a bank card as an example, the card image 601 is input into the field detection model 602 to obtain the field detection result of the card image. Figure 6 The field detection result includes the bank card validity period field area 603, the bank card number area 604, and the first half of the card number area 605.

[0101] It should be noted that, in the process of detecting the card image field of a bank card, the target field area is the bank card number area, and the attribute identifier is the first half of the card number area.

[0102] The field detection model locates the relative position of each field in the card image, and can determine the recognition direction of the text based on the obtained target field area and attribute identification area.

[0103] The field detection model is obtained by training a fully convolutional neural network based on sample card images. It should be noted that the training samples in this application are a large number of card images obtained from different shooting angles or placement angles, and each card image is annotated with each field area detection frame, target field area position, and attribute identification position. In this way, based on the learning of the neural network, the characteristics of each card image can be learned. Therefore, for each card image input into the field detection model, its field position and attribute identification can be obtained.

[0104] In order to improve the detection efficiency, a method for locating the field position through a field detection model to predict the direction of a text field is also provided in the embodiment of the present application. The sample card image of the field detection model includes a number of card images at different angles, and the field area detection frame, the target field area and the attribute identifier of the target field area are marked in the sample card image.

[0105] See also Figure 7, which shows a schematic diagram of the annotation of the training samples of the field detection model. Taking a bank card as an example, Figure (a) shows a card-certificate image 701 corresponding to a bank card placed horizontally in the usual direction, Figure (b) shows a card-certificate image 702 corresponding to a horizontal bank card flipped 90 degrees clockwise, Figure (c) shows a card-certificate image 703 corresponding to a horizontal bank card flipped 180 degrees clockwise, and Figure (d) shows a card-certificate image 704 corresponding to a horizontal bank card flipped clockwise at an arbitrary angle. In the card-certificate image, the solid-line frame represents the annotation of each field of the current card-certificate image, and the dotted-line frame represents the annotation of the attribute identifier, that is, the annotation of the first half of the bank card number.

[0106] When training the fully convolutional neural network model, the corresponding mask truth value can be generated based on the known positions of each field in the training sample, and optimized using a classification loss function, where the classification function includes but is not limited to BCEloss, Dice loss, etc.

[0107] Therefore, by detecting the bank card image based on the field detection model obtained from the above training samples, the corresponding bank card number field, the first half of the card number field and other field areas, such as the validity period field area, can be detected.

[0108] Specifically, for each detected mask, each connected domain can be obtained by calculating the connected domain algorithm, and then the minimum circumscribed rectangle of each connected domain can be used to obtain the detection frame of each field. The direction of the bank card or field can be determined by the predicted detection frame and the front part frame.

[0109] Correspondingly, in a possible implementation method provided in an embodiment of the present application, the attribute identification area and the non-attribute identification area of ​​the target field area are determined according to the attribute identification; a mask image is generated based on the attribute identification area and the non-attribute identification area; and according to the mask image, image mask processing is performed on the image of the target field area to obtain the recognition direction of the card image to be identified.

[0110] Still Figure 6 Take the example in for illustration, the attribute identification area is the first half of the bank card number area, the first half of the card number area can be marked as 1, and the non-first half area in the card number area can be marked as 0. You can get a mask image including 1 and 0, that is, a two-dimensional matrix array composed of 0 and 1, so that the four vertex coordinates and coordinate order of the card number position can be determined according to the value in the mask image, thereby determining the recognition direction of the card number field. It can also be understood that the direction from 1 to 0 is recorded as the recognition direction of the card number. Then, according to the recognition direction, the coordinates of the card number detection box can be converted into the coordinate order of normal recognition, so that the coordinate order of the detected field area when input into the text or character recognition model meets its recognition rules.

[0111] If the field detection model is not used to obtain the attribute identifier, it is necessary to perform feature recognition on each card image. That is, since the card image contains multiple fields, and the attribute meaning of each field is different, the field formats satisfied by each field are different. At least part of the attribute identifier in the field information can be extracted according to the preset field format. For example, the preset field format can represent the field with the longest character string. Therefore, the content of the field with the longest character string in the card image can be used as the attribute identifier. Specifically, after determining the attribute identifier, part of the information therein can also be extracted.

[0112] After the specific location of each field is detected by the field detection model, the detection result obtained by the field detection model is input into the text recognition model to obtain the recognition result. The text recognition model is obtained based on OCR (Optical Character Recognition) technology, and recognizes all the character information corresponding to the card image.

[0113] However, when character information is usually obtained, different recognition requirements may be required according to the format of different fields and the requirements of the final output result. In another embodiment of the present application, a method for obtaining recognition information is also provided. Figure 8 , which shows a flow chart of a method for obtaining identification information, the method comprising:

[0114] S801, identifying the card image according to the identification direction to obtain initial identification information;

[0115] Wherein, the initial identification information includes a field and field content matching the field;

[0116] S802, extract the initial identification information to obtain identification information.

[0117] The identification information represents the field content that matches the target field to be identified.

[0118] In an implementation, irrelevant fields or information may be filtered out. Fig. 9, which shows a schematic diagram of a scenario for obtaining identification information. This application scenario corresponds to a scenario in which a community information management system collects information on the social security cards of community residents. The fields of collected information that the information collection terminal 901 needs to obtain include: name, social security card number, and validity period field. The field content of the name field identified by the prior art is "Name Zhang San". If the identified content is directly uploaded to the information recording page of the information collection terminal 901, the recorded information will be "Name Name Zhang San", which does not conform to the format of the information record, and the redundant fields need to be deleted. In the embodiment of the present application, the social security card image 902 to be identified can be directly extracted from the identified content according to the target field to be identified "Name", and the field content "Zhang San" of the field can be uploaded to the information collection terminal 901, avoiding the re-processing of the identification information and saving processing time.

[0119] The following is an explanation of the card information recognition method provided by the present application in a specific implementation manner. When information recognition is performed on a card image, it can be processed through three neural networks, including a card positioning network 1001, a field positioning network 1002, and a recognition network 1003. Among them, the card positioning network 1001 is used to detect the card and rotate the non-horizontal card to the horizontal direction, that is, the card positioning network realizes the detection of the card image; the field positioning network 1002 can realize the positioning of the relative position of each field in the card and obtain the predicted text field direction; the recognition network 1003 is used to identify the content of each detected field and obtain the recognition information. For the specific implementation process of the above network, please refer to the specific description in the above embodiment, which will not be repeated here. Using the three neural networks in this embodiment for processing can make the processing process simpler and improve the processing efficiency and accuracy.

[0120] Another implementation method of detecting attribute identification in the embodiment of the present application may include:

[0121] Determine the target area of ​​the card image to be identified;

[0122] At least part of the attribute identification is obtained in the field of the card image to be identified according to the position correspondence between the field in the card image to be identified and the target feature area.

[0123] For example, see Fig.11, which shows a schematic diagram of identification of ID card information. Obtain a card image 1101 for identification of an ID card, and the card image 1101 represents an image of the front of the ID card. Since the card image 1101 includes a resident's ID card photo 1102, it is used as the basis for identification, and the character string closest to the ID card photo 1102 and the longest character string is used as an attribute identifier, that is, the attribute identifier citizen ID card number 1103, and then determine the field at one end of the ID card number closest to the ID card photo 1102, that is, the end field 1104 of the ID card number 1103, and then the direction opposite to the direction from the character at the end of the end field 1104 to the other end of the ID card number is used as the identification direction 1105, and then based on the identification direction, obtain identification information 1106, and the corresponding identification information 1106 may include: name field and content, gender field and content, ethnic field and content, date of birth field and content, address field and content, citizen ID card number field and content.

[0124] On the other hand, the present application also provides a card information recognition device, as shown in Fig.12 , which shows a schematic diagram of the composition of an embodiment of a card information recognition device of the present application. The device of this embodiment can be applied to a terminal or a server, and the device may include:

[0125] The image acquisition unit 1201 is used to acquire the card image to be identified;

[0126] The identification detection unit 1202 is used to detect at least part of the attribute identification in the card image;

[0127] A direction acquisition unit 1203, configured to obtain the recognition direction of the card image to be recognized according to the attribute identifier;

[0128] The information recognition unit 1204 is used to recognize the card image according to the recognition direction to obtain recognition information.

[0129] In a possible case, the image acquisition unit includes:

[0130] An image acquisition subunit, used to acquire the original image of the card to be identified;

[0131] The area recognition subunit is used to perform feature area recognition on the original image of the card to be recognized to obtain the central area of ​​the card to be recognized;

[0132] A region generating subunit, configured to generate an image acquisition region of the original image according to the central region, wherein the image acquisition region includes the attribute identifier;

[0133] The image acquisition subunit is used to acquire an image of the image acquisition area of ​​the original image to obtain a card image to be identified.

[0134] Optionally, the identification detection unit is specifically used to:

[0135] The card image is input into a field detection model to obtain attribute identifiers of a field area and a target field area of ​​the card image, wherein the target field area represents a field area having a preset field format, and the attribute identifier represents a front field area of ​​the target field area.

[0136] In a possible implementation, the device further includes:

[0137] A sample acquisition unit, used to acquire sample card images, wherein the sample card images include card images at several different angles, and the sample card images are annotated with a field area detection frame, a target field area, and an attribute identifier of the target field area;

[0138] The network training unit is used to train the fully convolutional neural network through the sample card image to obtain a field detection model.

[0139] Optionally, the direction acquiring unit is specifically used to:

[0140] Determine the attribute identification area and the non-attribute identification area of ​​the target field area according to the attribute identification;

[0141] Generate a mask image based on the attribute identification area and the non-attribute identification area;

[0142] According to the mask image, the image of the target field area is subjected to image mask processing to obtain the recognition direction of the card image to be recognized.

[0143] Optionally, the information identification unit includes:

[0144] An initial information recognition subunit, used to recognize the card image according to the recognition direction to obtain initial recognition information, wherein the initial recognition information includes fields and field contents;

[0145] The information extraction subunit is used to extract information from the initial identification information to obtain identification information, wherein the identification information represents the field content that matches the target field to be identified.

[0146] In a possible implementation manner, the region identification subunit is specifically used to:

[0147] Performing feature region recognition on the original image of the card to be recognized to obtain a plurality of feature sub-regions;

[0148] If a group of target characteristic sub-regions matching the card to be identified are screened out from the plurality of characteristic sub-regions, a central region of the card to be identified is acquired from the target sub-characteristic sub-regions.

[0149] In yet another possible implementation, the identification detection unit further includes:

[0150] A target area determination subunit, used to determine the target feature area of ​​the card image to be identified;

[0151] The identification information acquisition subunit is used to obtain at least part of the attribute identification in the field of the card image to be identified according to the position correspondence between the field in the card image to be identified and the target feature area.

[0152] On the other hand, the present application also provides a terminal, as shown in Fig.13 , which shows a schematic diagram of the composition structure of the terminal of the present application. The terminal 1300 of this embodiment may include: a processor 1301 and a memory 1302.

[0153] Optionally, the terminal may further include a communication interface 1303 , an input unit 1304 , a display 1305 , and a communication bus 1306 .

[0154] The processor 1301 , the memory 1302 , the communication interface 1303 , the input unit 1304 , and the display 1305 all communicate with each other via the communication bus 1306 .

[0155] In the embodiment of the present application, the processor 1301 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a readily available programmable gate array, or other programmable logic devices.

[0156] The processor may call the program stored in the memory 1302. Specifically, the processor may execute the operations executed by the application server side in the following embodiment of the message sending method.

[0157] The memory 1302 is used to store one or more programs. The program may include program code, and the program code includes computer operation instructions. In the embodiment of the present application, the memory at least stores a program for implementing the following functions:

[0158] Obtain the card image to be identified;

[0159] Detecting at least part of the attribute identifier in the card image;

[0160] According to the attribute identifier, obtaining the recognition direction of the card image to be recognized;

[0161] The card image is identified according to the identification direction to obtain identification information.

[0162] In one possible implementation, the memory 1302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function (such as an image acquisition function, etc.); the data storage area may be based on data created during the use of the computer, such as relevant data of a card detection model, etc.

[0163] In addition, the memory 1302 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0164] The communication interface 1303 may be an interface of a communication module, such as an interface of a GSM module.

[0165] The present application may further include a display 1304 and an input unit 1305 and the like.

[0166] certainly, Fig.13 The structure of the terminal shown does not constitute a limitation on the terminal in the embodiment of the present application. In actual applications, the terminal may include Fig.13 More or fewer components than shown, or combinations of certain components.

[0167] On the other hand, an embodiment of the present application further provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the card information identification method executed by the server side in any of the above embodiments is implemented.

[0168] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0169] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0170] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0171] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A card information recognition method, characterized in that: The method includes: Obtain the card image to be identified; Input the card image into a field detection model to obtain attribute identifiers of a field area and a target field area of ​​the card image, wherein the target field area represents a field area having a preset field format, and the attribute identifier represents a front part of the field area of ​​the target field area; Determine the attribute identification area and the non-attribute identification area of ​​the target field area according to the attribute identification; Generate a mask image based on the attribute identification area and the non-attribute identification area; According to the mask image, the image of the target field area is subjected to image mask processing to obtain the recognition direction of the card image to be recognized; the recognition direction includes the direction of any angle; Recognize the card image according to the recognition direction of the text collection frame matching the recognition direction to obtain recognition information; The training process of the field detection model includes: Acquire sample card images, wherein the sample card images include card images at several different angles, and the sample card images are marked with a field area detection frame, a target field area, and an attribute identifier of the target field area; The fully convolutional neural network is trained using the sample card images to obtain a field detection model.

2. The method according to claim 1, characterized in that The step of obtaining the card image to be identified includes: Obtain the original image of the card to be identified; Performing feature area recognition on the original image of the card to be identified to obtain the central area of ​​the card to be identified; Generate an image acquisition area of ​​the original image according to the central area, wherein the image acquisition area includes the attribute identifier; The image acquisition area of ​​the original image is subjected to image acquisition to obtain a card image to be identified.

3. The method according to claim 1, characterized in that: The step of identifying the card image according to the identification direction to obtain identification information includes: Recognize the card image according to the recognition direction to obtain initial recognition information, wherein the initial recognition information includes fields and field contents; Information extraction is performed on the initial identification information to obtain identification information, wherein the identification information represents field content that matches the target field to be identified.

4. The method according to claim 2, characterized in that: The step of performing feature area recognition on the original image of the card to be identified to obtain the central area of ​​the card to be identified includes: Performing feature region recognition on the original image of the card to be recognized to obtain a plurality of feature sub-regions; If a group of target feature sub-regions matching the card to be identified are screened out from the plurality of feature sub-regions, a central region of the card to be identified is obtained from the target feature sub-regions.

5. The method according to claim 1, characterized in that The detecting at least part of the attribute identification of the card image includes: Determining a target feature area of ​​the card image to be identified; At least part of the attribute identification is obtained in the field of the card image to be identified according to the position correspondence between the field in the card image to be identified and the target feature area.

6. A card information recognition device, characterized in that: The device includes: An image acquisition unit, used to acquire the card image to be identified; an identification detection unit, used for inputting the card image into a field detection model to obtain attribute identifications of a field area and a target field area of ​​the card image, wherein the target field area represents a field area having a preset field format, and the attribute identification represents a front part of the field area of ​​the target field area; A direction acquisition unit is used to determine the attribute identification area and the non-attribute identification area of ​​the target field area according to the attribute identification; generate a mask image based on the attribute identification area and the non-attribute identification area; perform image mask processing on the image of the target field area according to the mask image to obtain the recognition direction of the card image to be recognized; the recognition direction includes the direction of any angle; An information recognition unit, used to recognize the card image according to the recognition direction to obtain recognition information; A sample acquisition unit, used to acquire sample card images, wherein the sample card images include card images at several different angles, and the sample card images are annotated with a field area detection frame, a target field area, and an attribute identifier of the target field area; The network training unit is used to train the fully convolutional neural network through the sample card image to obtain a field detection model.

7. The device according to claim 6, wherein the image acquisition unit comprises: An image acquisition subunit, used to acquire the original image of the card to be identified; The area recognition subunit is used to perform feature area recognition on the original image of the card to be recognized to obtain the central area of ​​the card to be recognized; A region generating subunit, configured to generate an image acquisition region of the original image according to the central region, wherein the image acquisition region includes the attribute identifier; The image acquisition subunit is used to acquire an image of the image acquisition area of ​​the original image to obtain a card image to be identified.

8. The device according to claim 6, wherein the information identification unit comprises: An initial information recognition subunit, used to recognize the card image according to the recognition direction to obtain initial recognition information, wherein the initial recognition information includes fields and field contents; The information extraction subunit is used to extract information from the initial identification information to obtain identification information, wherein the identification information represents the field content that matches the target field to be identified.

9. The device according to claim 7, wherein the region identification subunit is specifically configured to: Performing feature region recognition on the original image of the card to be recognized to obtain a plurality of feature sub-regions; If a group of target feature sub-regions matching the card to be identified are screened out from the plurality of feature sub-regions, a central region of the card to be identified is obtained from the target feature sub-regions.

10. The device according to claim 6, wherein the identification detection unit further comprises: A target area determination subunit, used to determine the target feature area of ​​the card image to be identified; The identification information acquisition subunit is used to obtain at least part of the attribute identification in the field of the card image to be identified according to the position correspondence between the field in the card image to be identified and the target feature area.

11. A terminal, characterized in that: include: Processor and memory; Wherein, the memory is used to store programs; The processor is used to execute a program stored in the memory, wherein the program is at least used to: Obtain the card image to be identified; Input the card image into a field detection model to obtain attribute identifiers of a field area and a target field area of ​​the card image, wherein the target field area represents a field area having a preset field format, and the attribute identifier represents a front part of the field area of ​​the target field area; Determine the attribute identification area and the non-attribute identification area of ​​the target field area according to the attribute identification; Generate a mask image based on the attribute identification area and the non-attribute identification area; According to the mask image, the image of the target field area is subjected to image mask processing to obtain the recognition direction of the card image to be recognized; the recognition direction includes the direction of any angle; The card image is recognized according to the recognition direction of the text collection frame that matches the recognition direction to obtain recognition information.

12. A readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded and executed by a processor to implement each step of the card information identification method as claimed in any one of claims 1 to 5.

Images