Card text recognition method and device and electronic equipment

By preprocessing the card certificate image and identifying the text identification type, combined with the bidirectional long and short-term memory network, the problems of blurred and unclear boundaries in card certificate text recognition are solved, and the recognition accuracy is significantly improved.

CN119091429AInactive Publication Date: 2024-12-06华润数字科技有限公司 +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411197357.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing card certificate text recognition technology has problems such as vagueness and unclear boundaries, which leads to low recognition accuracy and the inability to conduct targeted detection of different card certificate text identification types.

Method used

A card certificate text recognition method is proposed, including obtaining target card certificate images, performing pre-processing (edge ​​processing, image segmentation and correction processing), identifying the card certificate text identification type, and text recognition based on the type, using a two-way long and short-term memory network Perform character detection and fusion.

Benefits of technology

Through edge detection, image segmentation and correction processing, the clarity and accuracy of card certificate images can be improved, and different card certificate text identification types can be identified, which significantly improves the accuracy of card certificate text recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091429A_ABST
    Figure CN119091429A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a card text recognition method and device and electronic equipment, and belongs to the field of image detection. The method comprises the following steps: acquiring a target card image; preprocessing the target card image to obtain a preprocessed card image; the preprocessing comprises at least one of edge processing, image segmentation and correction processing; and performing text recognition on the preprocessed card image according to the card text identification type. According to the embodiment of the invention, at least one operation of edge detection, image segmentation and correction processing is carried out on the card image, the card image with clear and continuous edges and / or after correction can be generated, and the accuracy of card text recognition can be improved subsequently; card text identification is carried out according to different card text identification types, the card text can be identified according to different card text identification types, and the accuracy of card text identification is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image detection, and in particular to a card text recognition method, device and electronic equipment. Background Art

[0002] At present, in the field of financial applications, it usually involves mobile payment, social security card account opening and bank card account opening scenarios, which require verification of the user's ID card or identification of the user's bank card or social security card to determine the user's ID card information, bank card information or social security card information.

[0003] In the prior art, card text recognition often directly captures card images, extracts features and performs text recognition on the card images, and converts the text on the card into recognizable text information; however, the card images detected by this method are fuzzy and have unclear boundaries, and cannot perform targeted detection for different card text identification types, resulting in a low accuracy rate for card text recognition. Therefore, how to improve the accuracy rate of card text recognition has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a card text recognition method, device and electronic device, aiming to improve the efficiency of card text recognition.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a card text recognition method, the method comprising:

[0006] Get the target card image;

[0007] Preprocessing the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing;

[0008] Performing text identification category recognition on the pre-processed card image to obtain the card text identification type;

[0009] The pre-processed card image is subjected to text recognition according to the card text identification type to obtain a target card text.

[0010] In some embodiments, if the pre-processing includes a combination of edge processing, image segmentation and rectification processing;

[0011] The preprocessing of the target card image to obtain a preprocessed card image includes:

[0012] Performing edge processing on the target card image to obtain an edge card image;

[0013] Performing image segmentation on the edge card image to obtain a segmented card image;

[0014] The segmented card image is corrected to obtain the pre-processed card image.

[0015] In some embodiments, performing edge processing on the target card image to obtain an edge card image includes:

[0016] Obtaining the initial pixel value of the target card image;

[0017] Performing a grayscale weighted average operation on the initial pixel value to obtain a grayscale pixel value;

[0018] Calculating the gradient amplitude and gradient direction of the grayscale pixel value to obtain a gradient pixel value;

[0019] The gradient pixel value is subjected to non-maximum suppression processing to obtain the edge card image.

[0020] In some embodiments, the segmented card image includes a foreground card image and a background card image;

[0021] The performing image segmentation on the edge card image to obtain a segmented card image includes:

[0022] Divide the edge card image into regions based on a preset initial image segmentation threshold to obtain an initial foreground region and an initial background region;

[0023] Obtaining a foreground pixel mean value and a foreground pixel allocation probability of the initial foreground area, and obtaining a background pixel mean value and a background pixel allocation probability of the initial background area;

[0024] Performing threshold screening on the initial image segmentation threshold based on the foreground pixel mean, the foreground pixel allocation probability, the background pixel mean and the background pixel allocation probability to obtain a target segmentation threshold;

[0025] An initial foreground area that is less than or equal to the target segmentation threshold is used as the foreground card image, and an initial background area that is greater than the target segmentation threshold is used as the background card image.

[0026] In some embodiments, the step of performing correction processing on the segmented card image to obtain the pre-processed card image includes:

[0027] Performing straight line detection on the segmented card image to obtain an initial image straight line;

[0028] Obtaining a straight line inclination angle of the straight line of the initial image;

[0029] Constructing a target correction matrix according to the segmented card image; the target correction matrix is ​​used to represent the correction position of the segmented card image;

[0030] The segmented card image is corrected according to the target correction matrix and the straight line inclination angle to obtain the pre-processed card image.

[0031] In some embodiments, the card text identification type includes a printing type and an embossing type;

[0032] The performing text identification category recognition on the pre-processed card image to obtain the card text identification type includes:

[0033] The pre-processed card image is spatially transformed according to a preset first color space to obtain a first color card image; the first color space is used to represent the color space of the pre-processed card image through hue, saturation and brightness;

[0034] Performing black pixel ratio detection on the first color card image to obtain the black pixel ratio of the image;

[0035] The card text identification type is determined according to the black pixel ratio of the image.

[0036] In some embodiments, after performing text identification category recognition on the pre-processed card image to obtain the card text identification type, and before performing text recognition on the pre-processed card image according to the card text identification type to obtain the target card text, the method further includes:

[0037] Convert the concave-convex type pre-processed card image into a preset second color space to obtain a second color card image; the second color space is used to represent the color space of the pre-processed card image through brightness, blue chromaticity and red chromaticity;

[0038] Acquire a second color image pixel value of the second color card image, and acquire a preset structure element; the structure element is used to represent the expansion range of the second color card image;

[0039] The second color card image is expanded according to the pixel value of the second color image and the structural element to obtain an expanded preprocessed card image.

[0040] In some embodiments, performing text recognition on the pre-processed card image according to the card text identification type to obtain the target card text includes:

[0041] Inputting the pre-processed card image into a pre-trained bidirectional long short-term memory network according to the card text identification type; the bidirectional long short-term memory network includes a forward output layer and a backward output layer;

[0042] Perform forward text character detection on the pre-processed card image through the forward output layer to obtain a forward time-series character sequence;

[0043] Performing backward text character detection on the pre-processed card image through the backward output layer to obtain a backward time-sequential character sequence;

[0044] The forward time sequence character sequence and the backward time sequence character sequence are merged to obtain the target card text.

[0045] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a card text recognition device, the device comprising:

[0046] A card image acquisition module, used to acquire a target card image;

[0047] An image preprocessing module, used to preprocess the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing;

[0048] A type identification module, used to identify the text identification category of the pre-processed card image to obtain the card text identification type;

[0049] The card text recognition module is used to perform text recognition on the pre-processed card image according to the card text identification type to obtain the target card text.

[0050] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0051] The card text recognition method, device and electronic device proposed in the present application obtain a target card image; pre-process the target card image to obtain a pre-processed card image; wherein the pre-processing includes at least one of edge processing, image segmentation and correction processing; identify the text identification category of the pre-processed card image to obtain the card text identification type; perform text recognition on the pre-processed card image according to the card text identification type to obtain the target card text. The present application can realize edge detection, background segmentation and / or image correction of the card image by performing at least one of edge detection, image segmentation and correction processing on the card image, generate a card image with clear edges, continuous and / or corrected, which is helpful to improve the accuracy of card text recognition in the future; and perform text recognition on the pre-processed card image according to the card text identification type, so as to identify the card text for different card text identification types, thereby improving the accuracy of card text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a card text recognition method provided by an embodiment of the present application;

[0053] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0054] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.

[0055] Figure 4 yes Figure 2 Flow chart of step S202 in FIG.

[0056] Figure 5 yes Figure 2 Flow chart of step S203 in FIG.

[0057] Figure 6 yes Figure 1 Flow chart of step S103 in FIG.

[0058] Figure 7 is another flow chart of the card text recognition method provided by the embodiment of the present application;

[0059] Figure 8 yes Figure 1 Flow chart of step S104 in FIG.

[0060] Fig. 9 This is an application example diagram of the card text recognition method provided in the embodiment of the present application;

[0061] Fig.10 It is a structural schematic diagram of a card text recognition device provided in an embodiment of the present application;

[0062] Fig.11 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Image Capt ion: Generate natural language descriptions for images and use the generated descriptions to help applications understand the semantics expressed in the image visual scene. For example, image description can convert image retrieval into text retrieval, which is used to classify images and improve image retrieval results. People usually only need to take a quick glance to describe the details of the image visual scene, while automatically adding descriptions to images is a comprehensive and arduous computer vision task that requires converting the complex information contained in the image into natural language descriptions. Compared with ordinary computer vision tasks, image captioning requires not only identifying objects from images, but also associating the identified objects with natural semantics and describing them in natural language. Therefore, image description requires people to extract deep features of images, associate with semantic features and convert them for generating descriptions.

[0068] Bi-directional long short-term memory network (Bi-LSTM): is a special type of recurrent neural network (RNN) that combines the memory unit and gating mechanism of LSTM with the ability to process data bidirectionally. Bi-LSTM consists of two LSTMs, forward and backward, which process the forward and reverse information of sequence data respectively, thereby capturing more comprehensive contextual information.

[0069] ResNet18: is a deep residual network, one of the ResNet series models. ResNet18 consists of 18 weight layers, including convolutional layers and fully connected layers, but excluding batch normalization layers (BN) and pooling layers. ResNet18 solves the gradient vanishing and gradient exploding problems that occur as the network depth increases by introducing residual learning, allowing the network to effectively train deeper hierarchical structures.

[0070] Based on this, the embodiments of the present application provide a card text recognition method, device and electronic device, aiming to improve the efficiency of card text recognition.

[0071] The card text recognition method, device and electronic device provided in the embodiments of the present application are specifically described through the following embodiments. First, the card text recognition method in the embodiments of the present application is described.

[0072] The card text recognition method provided in the embodiment of the present application relates to the field of image detection. The card text recognition method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the card text recognition method, etc., but is not limited to the above forms.

[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] See also Figure 1 , Figure 1 is a flowchart of a card text recognition method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0075] Step S101, obtaining a target card image.

[0076] Step S102, preprocessing the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing.

[0077] Step S103, performing text identification on the pre-processed card image to obtain the card text identification type.

[0078] Step S104, performing text recognition on the pre-processed card image according to the card text identification type to obtain the target card text.

[0079] Steps S101 to S104 shown in the embodiment of the present application are as follows: acquiring a target card image; preprocessing the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing; identifying the text identification category of the preprocessed card image to obtain the card text identification type; and performing text recognition on the preprocessed card image according to the card text identification type to obtain the target card text. The present application can realize edge detection, background segmentation and / or image correction of the card image by performing at least one of edge detection, image segmentation and correction processing on the card image, generating a card image with clear edges, continuous and / or corrected, which is helpful to improve the accuracy of card text recognition in the future; and performing text recognition on the preprocessed card image according to the card text identification type, so as to identify the card text for different card text identification types and improve the accuracy of card text recognition.

[0080] In step S101 of some embodiments, the target card image is a card RGB color image determined based on an actual scene.

[0081] For example, ID card photos, bank card images, driver's license images, social security cards or medical insurance cards, etc.

[0082] In some embodiments, before step S102, the target card image can be trained using the YoloV7 algorithm to obtain a card detection model; and the target card image is input into the card detection model for position detection to determine the key point position information of the card in the target card image.

[0083] Specifically, use labeling tools (such as Label Img, CVAT, etc.) to label the collected target card images, and clearly mark the location and category of the card (such as identity card, bank card, etc.) during labeling, and use the location, category and target card image of the card to train the pre-trained card detection model, and continuously adjust the model parameters until the output of the model meets the loss function of the model, thereby obtaining a trained card detection model.

[0084] For example, if the card detection model is a bank card detection model, and an image containing a bank card is input to the model, the model will output: bounding box and key points. The bounding box contains the coordinates of the upper left corner (a1, b1) and lower right corner (a2, b2) of the bank card, and these two coordinates define the position of the bank card in the image; the key points include the coordinates of the upper left corner (a_photo1, b_photo1) and lower right corner (a_photo2, b_photo2) of the bank card image area, as well as the starting and ending point coordinates of key information such as the bank card number and verification code number.

[0085] In step S102 of some embodiments, preprocessing includes at least one of edge processing, image segmentation and correction processing. If preprocessing includes a combination of edge processing, image segmentation and correction processing, it means that edge processing, image segmentation processing and correction processing are performed on the image at the same time.

[0086] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0087] Step S201, edge processing is performed on the target card image to obtain an edge card image;

[0088] Step S202, performing image segmentation on the edge card image to obtain a segmented card image;

[0089] Step S203, correcting the segmented card image to obtain a pre-processed card image.

[0090] In step S201 of some embodiments, the edge card image is an image composed of pixel values ​​after edge detection. For example, if the target card image is a bank card image, edge detection is performed on the bank card image to obtain an edge bank card image composed of pixel values ​​after edge detection.

[0091] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S304:

[0092] Step S301, obtaining the initial pixel value of the target card image.

[0093] Step S302, performing a grayscale weighted average operation on the initial pixel value to obtain a grayscale pixel value.

[0094] Step S303, calculating the gradient amplitude and gradient direction of the grayscale pixel value to obtain a gradient pixel value.

[0095] Step S304: performing non-maximum suppression processing on the gradient pixel values ​​to obtain an edge card image.

[0096] In step S301 of some embodiments, the initial pixel value refers to the pixel value in the target card image. For example, if the target card image is a bank card image, the pixel value of the bank card image is extracted as the initial pixel value.

[0097] In step S302 of some embodiments, the grayscale pixel value refers to the grayscale image pixel value.

[0098] Specifically, the grayscale pixel value can be determined by the following formula:

[0099] I(x,y)=αI R +βI G +γI B

[0100] Where I(x,y) represents the grayscale pixel value corresponding to the pixel coordinate point (x,y) of the target card image; I R Indicates the initial pixel value of the R channel of the target card image; I G Indicates the initial pixel value of the G channel of the target card image; I B It represents the initial pixel value of the B channel of the target card image; α represents the grayscale weight factor of the initial pixel value of the R channel; β represents the grayscale weight factor of the initial pixel value of the G channel; γ represents the grayscale weight factor of the initial pixel value of the B channel.

[0101] In this embodiment, by performing a grayscale weighted average operation on the initial pixel values, the brightness of different regions in the image can be adjusted in a weighted average manner to visually enhance the contrast between the images, thereby making the target card text area more prominent.

[0102] In step S303 of some embodiments, the gradient pixel value is used to describe the gradient size of the grayscale pixel value; wherein the gradient is a vector having two properties: amplitude and direction.

[0103] Specifically, the gradient magnitude reflects the intensity of the change in the brightness of the grayscale pixel value, and the gradient direction reflects the direction in which the brightness of the grayscale pixel value changes fastest.

[0104] Specifically, the grayscale pixel value convolution operator can be obtained by the following formula:

[0105]

[0106] Among them, S x Represents the convolution operator of the grayscale pixel value in the x-axis direction; S y Represents the convolution operator in the y-axis direction of the grayscale pixel value.

[0107] Furthermore, for S x The calculation of the horizontal gradient can be expressed by the following formula:

[0108] P[x,y]=(f[x,y+1]-f[x,y]+f[x+1,y+1]-f[x+1,y]) / 2

[0109] Among them, P[x,y] represents the pair S x The first-order partial derivative of , that is, the horizontal gradient; x represents the grayscale pixel horizontal coordinate; x+1 represents the adjacent coordinates of the grayscale pixel horizontal coordinate x; y represents the grayscale pixel vertical coordinate; y+1 represents the adjacent coordinates of the grayscale pixel vertical coordinate; f[x,y] represents the grayscale pixel value of the pixel point (x,y); f[x,y+1] represents the grayscale pixel value of the adjacent pixel point (x,y+1) directly below the pixel point (x,y); f[x+1,y+1] represents the grayscale pixel value of the adjacent pixel point (x+1,y+1) located to the lower right of the pixel point (x,y); f[x+1,y] represents the grayscale pixel value of the adjacent pixel point (x+1,y) located to the immediate right of the pixel point (x,y).

[0110] Furthermore, for S y The calculation of the vertical gradient can be expressed by the following formula:

[0111] Q[x,y]=(f[x,y]-f[x+1,y]+f[x,y+1]-f[x+1,y+1]) / 2

[0112] Where Q[x,y] represents S y The first-order partial derivative of , that is, the gradient in the vertical direction; x represents the horizontal coordinate of the grayscale pixel; x+1 represents the adjacent coordinates of the horizontal coordinate of the grayscale pixel x; y represents the vertical coordinate of the grayscale pixel; y+1 represents the adjacent coordinates of the vertical coordinate of the grayscale pixel; f[x,y] represents the grayscale pixel value of the pixel point (x,y); f[x,y+1] represents the grayscale pixel value of the adjacent pixel point (x,y+1) directly below the pixel point (x,y); f[x+1,y+1] represents the grayscale pixel value of the adjacent pixel point (x+1,y+1) located to the lower right of the pixel point (x,y); f[x+1,y] represents the grayscale pixel value of the adjacent pixel point (x+1,y) located to the right of the pixel point (x,y).

[0113] Furthermore, the gradient amplitude can be determined by the following formula:

[0114]

[0115] Among them, M[x,y] represents the gradient amplitude of the pixel point (x,y); P[x,y] represents the gradient in the horizontal direction; Q[x,y] represents the gradient in the vertical direction.

[0116] Furthermore, the gradient direction can be determined by the following formula:

[0117] θ[x,y]=arctan(Q[x,y] / P[x,y])

[0118] Among them, θ[x,y] represents the gradient direction of the pixel point (x,y); P[x,y] represents the gradient in the horizontal direction; Q[x,y] represents the gradient in the vertical direction.

[0119] In this embodiment, by calculating the gradient amplitude and gradient direction of the grayscale pixel value, the rate of change of image brightness can be understood through the gradient amplitude, and the direction of change of image brightness can be understood through the gradient direction, thereby realizing rapid determination of image edges based on the rate and direction of change of image brightness, thereby improving the efficiency and accuracy of image edge detection.

[0120] In step S304 of some embodiments, specifically, non-maximum suppression is a technology in edge detection, which is used to eliminate pixel points that do not belong to the edge in the image; in the gradient pixel value, the edge is usually manifested as a point with a large gradient amplitude; because not all points with a large gradient amplitude are edge points, non-maximum suppression compares the gradient amplitude of each gradient pixel point with other gradient pixel points in its neighborhood, retains the point with the largest local gradient amplitude as the edge point, and sets other gradient pixel points to 0 or a smaller value, thereby removing the edge points and obtaining an edge card image.

[0121] Specifically, non-maximum suppression can be achieved through the Canny edge detection algorithm, which needs to determine the gradient direction of each pixel point, and compare the gradient amplitude of the pixel point with the gradient amplitudes of other points in the neighborhood of the pixel point along the gradient direction, and take the point with the largest local gradient amplitude as the edge point, and set other pixel points to 0 or a smaller value, thereby achieving non-maximum suppression.

[0122] In this embodiment, by performing non-maximum suppression processing on the gradient pixel values, the gradient pixel values ​​of non-edge points can be effectively suppressed, and only the gradient pixel values ​​of those that may be edge points are retained, making the image edges clearer and continuous, and achieving accurate detection of image edges, which helps to subsequently improve the accuracy of card text recognition.

[0123] In step S202 of some embodiments, the card image is segmented to include a foreground card image and a background card image.

[0124] The foreground card image refers to the area of ​​the card image that contains text information, and the background card image refers to the area of ​​the card image that does not contain text information.

[0125] For example, for a bank card image, the images of the card number area, the verification area, and the pattern area all belong to the foreground card image; and the image of the blank area around the bank card belongs to the background card image.

[0126] See also Figure 4 In some embodiments, step S202 may include but is not limited to steps S401 to S404:

[0127] Step S401, dividing the edge card image into regions based on a preset initial image segmentation threshold to obtain an initial foreground region and an initial background region.

[0128] Step S402, obtaining the foreground pixel mean and foreground pixel allocation probability of the initial foreground area, and obtaining the background pixel mean and background pixel allocation probability of the initial background area.

[0129] Step S403 , performing threshold screening on the initial image segmentation threshold based on the foreground pixel mean, foreground pixel allocation probability, background pixel mean and background pixel allocation probability to obtain a target segmentation threshold.

[0130] Step S404: taking the initial foreground area that is smaller than or equal to the target segmentation threshold as the foreground card image, and taking the initial background area that is larger than the target segmentation threshold as the background card image.

[0131] In step S401 of some embodiments, specifically, the preset initial image segmentation threshold refers to a threshold that can segment the edge card image into a foreground area and a background area, which can be represented by T0, and the value range of T0 is between grayscale values ​​0-255.

[0132] Specifically, the image pixel value of the edge card image is divided by T0; if the image pixel value is greater than T0, it means that the image pixel value belongs to the initial background area; if the image pixel value is less than or equal to T0, it means that the image pixel value belongs to the initial foreground area.

[0133] In step S402 of some embodiments, the foreground pixel mean refers to the average pixel value of all image pixels in the initial foreground area; the foreground pixel allocation probability refers to the probability value of each image pixel in the edge card image being divided into the initial foreground area.

[0134] Specifically, the background pixel mean refers to the average pixel value of all image pixels in the initial background area; the background pixel assignment probability refers to the probability value of each image pixel in the edge card image being assigned to the initial background area.

[0135] In step S403 of some embodiments, the target segmentation threshold refers to an optimal threshold for optimally segmenting the foreground and background of the edge card image.

[0136] Specifically, first, the variance of the foreground pixel mean and the background pixel mean can be calculated to obtain the initial variance; secondly, the initial variance, foreground pixel assignment probability, and background pixel assignment probability are calculated between classes to obtain the inter-class variance corresponding to the initial image segmentation threshold; finally, the initial image segmentation threshold corresponding to the maximum inter-class variance is selected from the inter-class variance as the target segmentation threshold.

[0137] Furthermore, the between-class means can be calculated by the following formula:

[0138] σ 2 =P1P2(m1-m2) 2

[0139] Among them, σ 2 represents the inter-class variance corresponding to the initial image segmentation threshold; P1 represents the probability of foreground pixel assignment; P2 represents the probability of background pixel assignment; m1 represents the mean value of foreground pixels; m2 represents the mean value of background pixels; (m1-m2) 2 represents the initial variance.

[0140] Specifically, the initial image segmentation threshold T0 based on the above formula that maximizes the inter-class variance is the target segmentation threshold T.

[0141] In this embodiment, the initial image segmentation threshold is threshold screened based on the foreground pixel mean, foreground pixel allocation probability, background pixel mean and background pixel allocation probability to obtain the target segmentation threshold. The optimal target segmentation threshold can be found by calculating the inter-class variance under different initial image segmentation thresholds, which helps to reduce the misclassification between the image foreground and background and improve the effect and accuracy of image segmentation.

[0142] In step S404 of some embodiments, the initial foreground area less than or equal to the target segmentation threshold T is used as the foreground card image, and the initial background area greater than the target segmentation threshold T is used as the background card image.

[0143] In step S203 of some embodiments, the pre-processed card image may be a corrected card image.

[0144] See also Figure 5In some embodiments, step S203 may also include but is not limited to steps S501 to S504:

[0145] Step S501, performing straight line detection on the segmented card image to obtain the straight lines of the initial image.

[0146] Step S502, obtaining the straight line inclination angle of the straight line in the initial image.

[0147] Step S503, constructing a target correction matrix according to the segmented card image; the target correction matrix is ​​used to represent the correction position of the segmented card image.

[0148] Step S504, correcting the segmented card image according to the target correction matrix and the straight line inclination angle to obtain a pre-processed card image.

[0149] In step S501 of some embodiments, the initial image straight lines refer to two horizontal straight lines and two vertical straight lines in the segmented card image.

[0150] Specifically, the ρ (distance)-θ (angle parameter) space in the polar coordinate system can be first selected through Hough transform, and a two-dimensional accumulator array can be created in the distance-angle parameter space; secondly, for each non-zero pixel point in the segmented card image, all possible straight lines corresponding to it in the ρ-θ parameter space are calculated, and voting is performed on the two-dimensional accumulator array, and the two-dimensional accumulator array is further traversed according to the set voting threshold to determine the local maximum points in the two-dimensional accumulator array that exceed the voting threshold, and these local maximum points are mapped to the distance-angle parameter space to form peak points. The set of these points corresponds to the straight line in the distance-angle parameter space; finally, according to the parameters (ρ, θ) of the peak point, the straight line corresponding to the parameters (ρ, θ) is determined in the image space as the initial image straight line.

[0151] In step S502 of some embodiments, the straight line inclination angle refers to the inclination angle of two horizontal straight lines in the segmented card image.

[0152] Specifically, since the two horizontal lines are represented by the distance ρ and the angle parameter θ, the θ values ​​of the two horizontal lines are obtained, and the tilt angle of the segmented card image is the difference between the selected straight line angle parameter θ value and the horizontal axis angle. Among them, since the two horizontal lines are parallel, the θ values ​​of the two horizontal lines are equal.

[0153] For example, if the value of the horizontal straight line θ that divides the card image is 4° and the horizontal axis angle is 0°, then the inclination angle of the horizontal straight line is 4°.

[0154] In step S503 of some embodiments, the target correction matrix is ​​used to represent the correction position of the segmented card image.

[0155] Specifically, the segmented corner pixel points of the four corners on the segmented card image are obtained, and the corrected corner pixel points of the four corners on the corrected card image are obtained; and a target correction matrix is ​​constructed according to the segmented corner pixel points and the corrected corner pixel points.

[0156] Specifically, the getPerspectiveTransform function in the OpenCV library can be used to calculate the segmentation corner pixels and the correction corner pixels to obtain the target correction matrix.

[0157] In step S504 of some embodiments, specifically, the width and height correction transformation of the segmented card image can be performed through the warpPerspective function according to the target correction matrix and the straight line inclination angle to obtain a corrected card image.

[0158] In this embodiment, the segmented card image is corrected according to the target correction matrix and the straight line inclination angle, so that the image correction can be achieved, and the image inclination is prevented from affecting the subsequent card text area recognition effect, so as to improve the accuracy of subsequent card text recognition.

[0159] In some embodiments, before step S103, the corrected card image can be cropped to separate the card text information portion and the background image, and the card text information is collected to form a card text information detection data set, and the card text information detection model is trained using the YoloV7 algorithm. The card text information can be located by the trained card text information detection model to preliminarily determine the card text area.

[0160] For example, for the corrected bank card image, the image can be cropped to distinguish the information including the bank card number from the background image, and the bank card number and verification code number in the bank card image can be located through the trained card text information detection model to obtain the text area including the bank card number and verification code number.

[0161] See also Figure 6 In some embodiments, step S103 includes but is not limited to steps S601 to S603:

[0162] Step S601, spatially transforming the pre-processed card image according to a preset first color space to obtain a first color card image; the first color space is used to represent the color space of the pre-processed card image, which is embodied by hue, saturation and brightness.

[0163] Step S602: Perform black pixel ratio detection on the first color card image to obtain the black pixel ratio of the image.

[0164] Step S603, determining the card text identification type according to the black pixel ratio of the image.

[0165] In step S601 of some embodiments, the card text identification type includes a printed type and an embossed type; for example, the card number identification type of a bank card can be a printed card number type or an embossed card number type.

[0166] Specifically, the first color space can be an HSV space, which refers to the three color channels of the image, namely H (hue), S (saturation), and V (value); wherein H represents the type of image color; S represents the vividness of the image color, the larger the value, the darker and brighter the color, and vice versa, the lighter and duller the color; V represents the brightness of the image color.

[0167] Specifically, space conversion can be performed using the following formula:

[0168]

[0169] V=max(R′,G′,B′)

[0170] R′=R / 255, G′=G / 255, B′=B / 255

[0171] Among them, H represents the hue value; S represents the saturation value; V represents the brightness value; R′ represents the red brightness or intensity of the pre-processed card image; G′ represents the green brightness or intensity of the pre-processed card image; B′ represents the blue brightness or intensity of the pre-processed card image; R represents the red value of the pre-processed card image; G represents the green value of the pre-processed card image; B represents the blue value of the pre-processed card image.

[0172] In this embodiment, the pre-processed card image is spatially transformed according to a preset first color space to obtain a first color card image, which can intuitively represent the color brightness, color tone and color vividness of the pre-processed card image, thereby facilitating the subsequent improvement of the recognition accuracy of the card text identification type.

[0173] In step S602 of some embodiments, the image melanin ratio refers to the ratio of black pixels in the first color card image, and the image melanin ratio is used to indicate the brightness of the first color card image.

[0174] Specifically, a brightness threshold of the component occupied by the brightness value V in the HSV preprocessed card image is set, and the brightness value of the first color card image is compared with the brightness threshold. If the brightness value is less than the brightness threshold, the image pixel of the first color card image is marked as a black pixel; the number of all black pixels is counted, and the number of all black pixels is divided by the total number of pixels of the first color card image to obtain the proportion of black pixels in the image.

[0175] For example, if the brightness threshold is 50, then if the brightness value image pixel value of the first color card image is less than 50, the image pixel value of the first color card image is regarded as 255, indicating that the image pixel is a black pixel. If the number of black pixels with a pixel value of 255 is 12,000, and the total number of pixels in the first color card image is 80,000, then the number of all black pixels 12,000 is divided by the total number of pixels in the first color card image 80,000, and the proportion of black pixels in the image is 15%.

[0176] In this embodiment, the black pixel ratio detection is performed on the first color card image. In the HSV space, since the lightness value V component in the HSV directly corresponds to the brightness of the color, and the brightness is the key factor in determining whether the color is black, by setting the brightness threshold of the V component, pixels close to black can be more accurately identified, thereby improving the accuracy of the black pixel ratio.

[0177] In step S603 of some embodiments, specifically, a preset black pixel ratio threshold is obtained, and the black pixel ratio threshold is compared with the black pixel ratio; if the black pixel ratio is greater than the black pixel ratio threshold, it indicates that the card text logo type is a printed type; if the black pixel ratio is less than or equal to the black pixel ratio threshold, it indicates that the card text logo type is a concave-convex type.

[0178] For example, the black pixel ratio threshold of the bank card number area is 20%. When the black pixel ratio of the bank card number area is greater than the black pixel ratio threshold of 20%, it indicates that the bank card number type is a printed card number; when the black pixel ratio of the bank card number area is less than or equal to the black pixel ratio threshold of 20%, it indicates that the bank card number type is a convex card number.

[0179] In this embodiment, the card text identification type is determined according to the proportion of black pixels in the image. In the HSV space, black pixels can be quickly screened out through simple threshold judgment without the need for complex color mixing calculations in the RGB space, thereby improving the efficiency of black pixel recognition. In addition, based on the proportion of black pixels, the card text identification type can be effectively distinguished, reducing the misjudgment or omission of subsequent card text recognition, thereby improving the accuracy of subsequent card text recognition.

[0180] See also Figure 7 In some embodiments, after step S103 and before step S104, the card text recognition method further includes steps S701 to S703:

[0181] Step S701, converting the concave-convex type pre-processed card image into a preset second color space to obtain a second color card image; the second color space is used to represent the color space of the pre-processed card image through brightness, blue chromaticity and red chromaticity.

[0182] Step S702, obtaining a second color image pixel value of the second color card image, and obtaining a preset structure element; the structure element is used to represent the expansion range of the second color card image.

[0183] Step S703: dilate the second color card image according to the pixel values ​​and the structural elements of the second color image to obtain a dilated pre-processed card image.

[0184] In step S701 of some embodiments, specifically, the second color space may be a YUV space, where the three color channels of the image are Y (Luminance, brightness value), U (Chrominance for Blue, blue chromaticity value), and V (Chrominance for Red, red chromaticity value); wherein Y represents the brightness information of the image, reflecting the intensity change of the image from the brightest to the darkest; U represents the chromaticity component of the blue in the image, reflecting the amount of blue in the image; and V represents the chromaticity component of the red in the image, reflecting the amount of red in the image.

[0185] Specifically, since the concave-convex type is greatly affected by the background interference of the pre-processed card image, the card background image needs to be further removed, which can be achieved by converting the pre-processed card image from the RGB color space to the YUV color space.

[0186] Specifically, the pre-processed card image can be converted into the second color space by the following formula:

[0187] Y=0.299R+0.587G+0.144B

[0188] U=-0.169R-0.331G+0.500B+128

[0189] V=0.500R-0.419G-0.081B+128

[0190] Among them, Y represents the brightness value; U represents the saturated blue chromaticity value; V represents the red chromaticity value; R represents the red value of the pre-processed card image; G represents the green value of the pre-processed card image; B represents the blue value of the pre-processed card image.

[0191] In this embodiment, by converting the concave-convex type pre-processed card image into a preset second color space, the brightness and chromaticity of the image can be separated. Since the concave-convex type is greatly disturbed by the background of the pre-processed card image, the edge image of the blue chromaticity component and the edge image of the red chromaticity component are calculated point by point to further remove the corrected card background image, which helps to improve the image quality of the pre-processed card image.

[0192] In step S702 of some embodiments, the structure element is a binary matrix defined by an image processing library (such as OpenCV, PI L, etc.); it includes a shape and a size, and the shape can be a square, a rectangle, a cross, a circle, etc.

[0193] Specifically, based on the shape and size of the structure element, the structure element can be used to represent the expansion range of the second color card image.

[0194] In step S703 of some embodiments, the expansion operation may be performed specifically by the following formula:

[0195]

[0196] Among them, A represents the second color card image; B represents the structural element; (x, y) represents the pixel coordinate point of the second color card image; x represents the horizontal coordinate of the pixel of the second color image; y represents the vertical coordinate of the pixel value of the second color image; (i, j) represents the pixel coordinate point of the structural element (equivalent to the center point or origin of the structural element); i represents the horizontal coordinate of the pixel of the structural element; j represents the vertical coordinate of the pixel of the structural element; A (x + i, y + j) represents the pixel value corresponding to the current position of the structural element B in the second color card image A.

[0197] In this embodiment, the second color card image is expanded according to the pixel value and the structural element of the second color image, so that the edge image can be made more continuous and obvious, and the accuracy of the detected image edge can be further improved. The expansion operation is combined with a suitable structural element size to remove small noise points in the image. For small holes or broken areas in the image, these areas can be connected through the expansion operation to obtain a complete image, thereby further improving the image quality of the pre-processed card image.

[0198] See also Figure 8 In some embodiments, step S104 may include but is not limited to steps S801 to S804:

[0199] Step S801: input the pre-processed card image into a pre-trained bidirectional long short-term memory network according to the card text identification type; the bidirectional long short-term memory network includes a forward output layer and a backward output layer.

[0200] Step S802, performing forward text character detection on the pre-processed card image through the forward output layer to obtain a forward time-series character sequence.

[0201] Step S803, performing backward text character detection on the pre-processed card image through the backward output layer to obtain a backward time-series character sequence.

[0202] Step S804: merge the forward time sequence character sequence and the backward time sequence character sequence to obtain the target card text.

[0203] In step S801 of some embodiments, the pre-trained bidirectional long short-term memory network is composed of a Bi-LSTM (Bi-directional Long Short-Term Memory) model, which mainly functions to capture the forward and backward dependencies between pre-processed card image features.

[0204] Among them, the bidirectional long short-term memory network includes: a forward output layer and a backward output layer; the forward output layer is used to detect the forward text characters of the pre-processed card image at each moment; the backward output layer is used to detect the backward text characters of the pre-processed card image at each moment.

[0205] In an optional embodiment of the present application, before the pre-processed card image is input into the bidirectional long short-term memory network, a deep residual operation is performed on the pre-processed card image through the ResNet18 network to extract image features of the pre-processed card image, and then the image features are input into the bidirectional long short-term memory network for card text recognition. The deep feature extraction capability of ResNet18 can be used to extract image features containing rich spatial information, and the bidirectional long short-term memory network is combined to process long-distance dependencies between characters, thereby enhancing the expression ability of the model and improving the accuracy of subsequent card text recognition.

[0206] In step S802 of some embodiments, the forward text character detection may be specifically expressed by the following formula:

[0207] h t =f(w1x t +w2h t-1 )

[0208] Among them, h t represents the forward sequential character sequence at time t; w1 represents the weight value of the forward text character at time t, w2 represents the weight value of the forward sequential character sequence at time t-1; x t represents the forward text character at time t; h t-1 Represents the forward sequential character sequence at time t-1.

[0209] In step S803 of some embodiments, the backward text character detection may be specifically expressed by the following formula:

[0210] h′ t =f(w3x′ t +w5h′ t-1 )

[0211] Among them, h′ t represents the backward sequential character sequence at time t; w3 represents the weight value of the backward text character at time t, w5 represents the weight value of the backward sequential character sequence at time t-1; x′ t represents the backward text character at time t; h′ t-1 Represents the backward sequential character sequence at time t-1.

[0212] In step S804 of some embodiments, the forward time sequence character sequence and the backward time sequence character sequence may be merged by the following formula:

[0213] o t =g(w4h t +w6h′ t )

[0214] Among them, t represents the target card text at time t; w4 represents the weight value of the forward sequential character sequence at time t, and w6 represents the weight value of the backward sequential character sequence at time t; h t Represents the forward time sequence of characters at time t; h′ t Represents the backward sequential character sequence at time t.

[0215] In this embodiment, during the card text recognition process, the bidirectional long short-term memory network needs to be trained through the CTC Loss function, and the Blank mechanism in the CTC Loss function can process redundant information in the output character sequence, that is, the Blank mechanism can distinguish the true boundary of the characters through the Blank symbol when the bidirectional long short-term memory network continuously predicts the same characters, so as to remove repeated characters during decoding. If the decoding process may involve continuous repeated characters, the continuous repeated characters will be merged, and the Blank symbol in the path will be removed to obtain the final target card text.

[0216] For example, among the five moments t0-t4, the characters mapped at moments t0, t1, and t2 are "a", and the characters mapped at moments t3 and t4 are "b", then the mapped characters are merged to obtain "aaabb", and then the consecutive repeated characters are merged and processed, and the final output character is "ab".

[0217] In this embodiment, the forward time sequence character sequence and the backward time sequence character sequence are fused, and the past and future context-related information can be processed simultaneously through a bidirectional long short-term memory network, which can better capture the dynamic characteristics and dependencies of the characters in the time sequence, thereby improving the accuracy of card text recognition.

[0218] See also Fig. 9 In an application scenario, taking the card text recognition of a bank card as an example, firstly, the card information is obtained by collecting and cleaning the bank card data; and the card information is trained using the YoloV7 algorithm to obtain a card detection model; the bank card image is input into the card detection model for position detection to determine the key point position information of the bank card in the bank card image; secondly, the detected bank card is edge processed, image segmented and corrected based on the key point position information, and finally a corrected bank card image is obtained; and the corrected bank card image is cropped to separate the bank card number part and the bank card background image, and the bank card number is collected to form a bank card number detection dataset, and YoloV7 is used to detect the bank card number. The loV7 algorithm is used to train the bank card number detection data set to obtain a bank card number detection model; further, the bank card number area is located using the trained bank card number detection model; the card number in the bank card number area is identified by the card number category to determine whether the bank card number category is an embossed card number type or a printed card number type; background removal and expansion operations are performed on the embossed card number type to extract a bank card number area with clear edges; the printed card number type bank card number is converted into HSV space, and the image pixels of the HSV space are used as the bank card number area; finally, the bank card number area is identified by bank card number to obtain the bank card number; and the bank card number is sent to the user end.

[0219] The embodiment of the present application obtains a target card image; performs edge processing on the target card image to obtain an edge card image; performs image segmentation on the edge card image to obtain a segmented card image; performs correction processing on the segmented card image to obtain a corrected card image; identifies the card text identification type of the corrected card image; performs text recognition on the corrected card image according to the card text identification type to obtain the target card text. The present application can realize edge detection, background segmentation and image correction of the card image by performing edge detection, image segmentation and correction processing operations on the card image, and generate a card image with clear edges, continuous and corrected, which is helpful to improve the accuracy of card text recognition in the future; and performs text recognition on the corrected card image according to the card text identification type, so as to identify the card text for different card text identification types, thereby improving the accuracy of card text recognition.

[0220] See also Fig.10The present application also provides a card text recognition device, which can implement the above card text recognition method, and the device includes:

[0221] A card image acquisition module, used to acquire a target card image;

[0222] An image preprocessing module, used to preprocess the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing;

[0223] A type recognition module is used to identify the text identification category of the pre-processed card image to obtain the card text identification type;

[0224] The card text recognition module is used to perform text recognition on the pre-processed card image according to the card text identification type to obtain the target card text.

[0225] The specific implementation of the card text recognition device is basically the same as the specific implementation of the card text recognition method described above, and will not be described in detail here.

[0226] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above card text recognition method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0227] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0228] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0229] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the card text recognition method of the embodiment of this application;

[0230] Input / output interface 1103, used to implement information input and output;

[0231] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0232] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0233] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0234] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0235] The card text recognition method, card text recognition device and electronic device provided in the embodiment of the present application obtain a target card image; pre-process the target card image to obtain a pre-processed card image; wherein the pre-processing includes at least one of edge processing, image segmentation and correction processing; identify the text identification category of the pre-processed card image to obtain the card text identification type; perform text recognition on the pre-processed card image according to the card text identification type to obtain the target card text. The present application can realize edge detection, background segmentation and / or image correction of the card image by performing at least one of edge detection, image segmentation and correction processing on the card image, generate a card image with clear edges, continuous and / or corrected, which is helpful to improve the accuracy of card text recognition in the future; and perform text recognition on the pre-processed card image according to the card text identification type, so as to identify the card text for different card text identification types, thereby improving the accuracy of card text recognition.

[0236] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0237] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0238] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0239] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0240] The terms "first", "second", "third", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0241] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0242] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0243] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0245] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0246] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A card text recognition method, It is characterized in that The method comprises: Get the target card image; Preprocessing the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing; Performing text identification category recognition on the pre-processed card image to obtain the card text identification type; The pre-processed card image is subjected to text recognition according to the card text identification type to obtain a target card text.

2. The method according to claim 1, It is characterized in that If the preprocessing includes a combination of edge processing, image segmentation and correction processing; The preprocessing of the target card image to obtain a preprocessed card image includes: Performing edge processing on the target card image to obtain an edge card image; Performing image segmentation on the edge card image to obtain a segmented card image; The segmented card image is corrected to obtain the pre-processed card image.

3. The method according to claim 2, It is characterized in that The step of performing edge processing on the target card image to obtain an edge card image includes: Obtaining the initial pixel value of the target card image; Performing a grayscale weighted average operation on the initial pixel value to obtain a grayscale pixel value; Calculating the gradient amplitude and gradient direction of the grayscale pixel value to obtain a gradient pixel value; The gradient pixel value is subjected to non-maximum suppression processing to obtain the edge card image.

4. The method according to claim 2, It is characterized in that The segmented card image includes a foreground card image and a background card image; The performing image segmentation on the edge card image to obtain a segmented card image includes: Divide the edge card image into regions based on a preset initial image segmentation threshold to obtain an initial foreground region and an initial background region; Obtaining a foreground pixel mean value and a foreground pixel allocation probability of the initial foreground area, and obtaining a background pixel mean value and a background pixel allocation probability of the initial background area; Performing threshold screening on the initial image segmentation threshold based on the foreground pixel mean, the foreground pixel allocation probability, the background pixel mean and the background pixel allocation probability to obtain a target segmentation threshold; An initial foreground area that is less than or equal to the target segmentation threshold is used as the foreground card image, and an initial background area that is greater than the target segmentation threshold is used as the background card image.

5. The method according to claim 2, It is characterized in that The correcting the segmented card image to obtain the pre-processed card image includes: Performing straight line detection on the segmented card image to obtain an initial image straight line; Obtaining a straight line inclination angle of the straight line of the initial image; Constructing a target correction matrix according to the segmented card image; the target correction matrix is ​​used to represent the correction position of the segmented card image; The segmented card image is corrected according to the target correction matrix and the straight line inclination angle to obtain the pre-processed card image.

6. The method according to claim 1, It is characterized in that The card text identification type includes a printed type and an embossed type; The performing text identification category recognition on the pre-processed card image to obtain the card text identification type includes: The pre-processed card image is spatially transformed according to a preset first color space to obtain a first color card image; the first color space is used to represent the color space of the pre-processed card image through hue, saturation and brightness; Performing black pixel ratio detection on the first color card image to obtain the black pixel ratio of the image; The card text identification type is determined according to the black pixel ratio of the image.

7. The method according to claim 6, It is characterized in that After performing text identification category recognition on the pre-processed card image to obtain the card text identification type, and before performing text recognition on the pre-processed card image according to the card text identification type to obtain the target card text, the method further includes: Convert the concave-convex type pre-processed card image into a preset second color space to obtain a second color card image; the second color space is used to represent the color space of the pre-processed card image through brightness, blue chromaticity and red chromaticity; Acquire a second color image pixel value of the second color card image, and acquire a preset structure element; the structure element is used to represent the expansion range of the second color card image; The second color card image is expanded according to the pixel value of the second color image and the structural element to obtain an expanded preprocessed card image.

8. The method according to any one of claims 1 to 7, It is characterized in that The performing text recognition on the pre-processed card image according to the card text identification type to obtain the target card text includes: Inputting the pre-processed card image into a pre-trained bidirectional long short-term memory network according to the card text identification type; the bidirectional long short-term memory network includes a forward output layer and a backward output layer; Perform forward text character detection on the pre-processed card image through the forward output layer to obtain a forward time-series character sequence; Performing backward text character detection on the pre-processed card image through the backward output layer to obtain a backward time-sequential character sequence; The forward time sequence character sequence and the backward time sequence character sequence are merged to obtain the target card text.

9. A card text recognition device, It is characterized in that The device comprises: A card image acquisition module, used to acquire a target card image; An image preprocessing module, used to preprocess the target card image to obtain a preprocessed card image; wherein the preprocessing includes at least one of edge processing, image segmentation and correction processing; A type identification module, used to identify the text identification category of the pre-processed card image to obtain the card text identification type; The card text recognition module is used to perform text recognition on the pre-processed card image according to the card text identification type to obtain the target card text.

10. An electronic device, It is characterized in that The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the card text recognition method according to any one of claims 1 to 8 when executing the computer program.