Information bearing medium identification method, identification device, computer equipment and medium

Abstract

The present invention discloses an information-bearing medium recognition method, recognition device, computer-readable storage device and computer device, the recognition method comprising: obtaining a first image of an information-bearing medium; performing text recognition on the first image to obtain text content of the information-bearing medium; classifying the information-bearing medium to obtain the type of the information-bearing medium; and text-archiving the text content according to the type. The embodiments provided by the present invention recognize different types of information-bearing media through an image classifier or a text classifier, which can make up for the problem of only recognizing a single type of information-bearing medium in the prior art, effectively improve image processing efficiency, and have broad application prospects.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an information bearing medium recognition method, a recognition device, a computer readable storage medium and a computer device. Background Art

[0002] With the development of image processing technology, the application of OCR technology to identify information-bearing media has become more common. The main form of OCR recognition applications on the market is to call the program with a mobile phone, then take a photo of the information-bearing medium and upload it to return the recognition result. This type of product has the following problems:

[0003] 1. Each type of information-bearing medium identification corresponds to a separate application (waybill identification, business card identification, ID card identification, etc.). If you want to identify different objects, you need to switch to different applications.

[0004] 2. Overcome the problem that the information-bearing medium is not capable of recognizing large-angle rotations and cannot automatically correct even small-angle rotations;

[0005] 3. When the background contains other text, it will be mistaken for the content of the information-bearing medium;

[0006] 4. When photographing an information-bearing medium, if the incident angle and the information-bearing medium are at a certain angle, the photographed information-bearing medium will have trapezoidal distortion, resulting in failure of text recognition. Summary of the invention

[0007] In order to solve at least one of the above problems, the present invention provides a method for identifying an information bearing medium in a first aspect, comprising:

[0008] acquiring a first image of an information bearing medium;

[0009] Performing text recognition on the first image to obtain text content of the information bearing medium;

[0010] classifying the information bearing medium to obtain a type of the information bearing medium;

[0011] The text content is filed according to the type.

[0012] Further, the classifying the information bearing medium to obtain the type of the information bearing medium includes:

[0013] roughly classifying the information bearing medium according to the aspect ratio of the first image;

[0014] The roughly classified information bearing medium is finely classified based on the first image using a preset corresponding image classifier to obtain the type of the information bearing medium.

[0015] Further, the classifying the information bearing medium to obtain the type of the information bearing medium includes:

[0016] roughly classifying the information bearing medium according to the aspect ratio of the first image;

[0017] The roughly classified information bearing medium is finely classified based on the text content using a preset corresponding text classifier to obtain the type of the information bearing medium.

[0018] Further, obtaining the first image of the information bearing medium includes:

[0019] Based on the acquired image, target detection and correction are performed on the information bearing medium in the image to acquire the first image;

[0020] Determine whether the first image meets the preset requirements, if not

[0021] Text direction detection is performed on the first image and the first image is rotated to correct the first image.

[0022] Furthermore, the performing target detection and correction on the information bearing medium in the image based on the acquired image to acquire the first image includes:

[0023] Perform image binarization based on the acquired image;

[0024] Performing edge detection to obtain the outline of the largest rectangle in the image;

[0025] The first image is segmented from the image by perspective transformation.

[0026] Furthermore, the performing target detection and correction on the information bearing medium in the image based on the acquired image to acquire the first image includes:

[0027] Perform image binarization based on the acquired image;

[0028] Performing straight line detection to obtain horizontal straight line sets and vertical straight line sets grouping, and merging approximately parallel lines to determine the optimal boundary and vertex of the information bearing medium;

[0029] The first image is segmented from the image by perspective transformation.

[0030] Further, the performing text direction detection on the first image and rotating the first image to correct the first image includes:

[0031] Using a preset full-angle text detection and classification model to perform text direction detection on the first image and output a rotation angle;

[0032] The first image is rotated by the rotation angle.

[0033] Furthermore, performing text recognition on the first image to obtain text content of the information bearing medium includes:

[0034] Performing text detection on the first image to obtain a plurality of second images;

[0035] The plurality of second images are recognized using a preset text recognition network model to obtain text content of the information bearing medium.

[0036] Further, performing text detection on the first image to obtain a plurality of second images includes:

[0037] Acquire an axisymmetric bounding box according to the first image;

[0038] Performing text score prediction on each of the axisymmetric bounding boxes to obtain a tilted minimum area box;

[0039] Detect the small text in the tilted minimum area frame and acquire a second image.

[0040] Further, the textual archiving of the text content according to the type includes:

[0041] According to the type, the text content is archived using a preset corresponding text archiving template and the archiving information of the information bearing medium is obtained.

[0042] Furthermore, after acquiring the first image of the information carrying medium and before performing text recognition on the first image to acquire the text content of the information carrying medium, the method further includes:

[0043] Perform barcode detection on the first image.

[0044] Further, the performing barcode detection on the first image includes:

[0045] performing image rotation correction on the first image;

[0046] Performing noise reduction processing on the corrected first image;

[0047] Performing edge detection on the first image after noise reduction to obtain an edge image;

[0048] Probabilistic transformation is used to perform line segment detection and filtering on the edge image to determine whether a barcode exists. If not, the barcode detection is exited. If yes, the line segment at the edge of the barcode is horizontally expanded to obtain a connected domain, the barcode area of ​​the barcode is determined based on the connected domain, and decoding is performed to identify the information bearing medium.

[0049] A second aspect of the present invention provides an information bearing medium identification device, comprising:

[0050] A distortion correction module, used to obtain a first image of the information bearing medium;

[0051] A text recognition module, used for performing text recognition on the first image to obtain text content of the information bearing medium;

[0052] An information bearing medium classification module, used for classifying the information bearing medium to obtain the type of the information bearing medium;

[0053] The text archiving module is used to archive the text content according to the type.

[0054] Furthermore, it also includes:

[0055] A barcode detection module is used to perform barcode detection on the first image.

[0056] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information bearing medium identification method as described in the first aspect.

[0057] A fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0058] The beneficial effects of the present invention are as follows:

[0059] In view of the current existing problems, the present invention develops an information-bearing medium recognition method, recognition device, computer-readable storage device and computer device. The image classifier or text classifier recognizes different types of information-bearing media, which can make up for the problem of only recognizing a single type of information-bearing medium in the prior art, effectively improve the image processing efficiency, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0061] Figure 1 A flow chart showing an identification method according to an embodiment of the present invention;

[0062] Figure 2 A flowchart showing distortion correction according to an embodiment of the present invention is shown;

[0063] Figure 3 A flowchart showing distortion correction according to rectangle detection according to one embodiment of the present invention;

[0064] Figure 4a-4b A schematic diagram showing a phased diagram of distortion correction for rectangle detection according to an embodiment of the present invention;

[0065] Figure 5 A flow chart showing distortion correction according to straight line detection according to one embodiment of the present invention;

[0066] Figure 6a-6b A schematic diagram showing the phased distortion correction of straight line detection according to an embodiment of the present invention is shown;

[0067] Figure 7 A flowchart showing text recognition according to an embodiment of the present invention;

[0068] Figure 8 A flowchart showing text detection according to an embodiment of the present invention is shown;

[0069] Figure 9a-9c A schematic diagram showing the phased process of text detection according to an embodiment of the present invention is shown;

[0070] Fig.10 A flowchart showing image classification according to an embodiment of the present invention is shown;

[0071] Fig.11 A flowchart showing text classification according to an embodiment of the present invention is shown;

[0072] Figure 12a-12b A schematic diagram showing the stages of text archiving according to an embodiment of the present invention;

[0073] Figures 13a-13c A schematic diagram showing the stages of barcode detection according to an embodiment of the present invention is shown;

[0074] Fig.14 A structural block diagram showing an information bearing medium identification device according to an embodiment of the present invention;

[0075] Fig.15 A schematic structural diagram of a computer device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0076] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0077] like Figure 1As shown, an embodiment of the present invention provides a method for identifying an information-carrying medium, comprising: acquiring a first image of an information-carrying medium; performing text recognition on the first image to acquire text content of the information-carrying medium; classifying the information-carrying medium to acquire a type of the information-carrying medium; and text-archiving the text content according to the type.

[0078] In a specific example, Figure 1 As shown, taking business card recognition as an example, the recognition method specifically includes:

[0079] First, a first image of the information bearing medium is acquired.

[0080] An image including a business card is acquired. The business card in the image may be in an abnormal mode. For example, the business card is in a deformed state in the image. In this case, the text in the business card is also in a deformed state and is difficult to recognize. Alternatively, characters or other text may exist in the background of the business card in the image. These characters or other text may be mistaken for the text in the business card.

[0081] To solve the above problems, Figure 2 As shown, in an optional embodiment, the obtaining of the first image of the information-bearing medium includes: performing target detection and correction on the information-bearing medium in the image based on the obtained image to obtain the first image; determining whether the first image meets preset requirements, and if not, performing text direction detection on the first image and rotating it to correct the first image.

[0082] That is, the business card itself is first accurately identified from the acquired image, the business card is segmented from the image, and the quadrilateral is stretched according to the actual proportion of the business card to obtain the first image to achieve distortion correction of the acquired image, and the deformed text in the business card is restored, thereby solving the problem of misrecognition of text due to the business card being in a deformed state or the presence of characters or text in the background.

[0083] In this embodiment, target detection is provided to identify the business card image, for example, by correcting the rectangular target detection, such as Figure 3 As shown, in an optional embodiment, the target detection and correction of the information bearing medium in the image based on the acquired image to obtain the first image includes: image binarization based on the acquired image; edge detection to obtain the outline of the largest rectangle in the image; and segmenting the first image from the image through perspective transformation.

[0084] Specifically, Figure 4a As shown, the image including the deformed business card is distorted:

[0085] First, the image is binarized based on the acquired image.

[0086] Then, edge detection is performed to obtain the outline of the largest rectangle in the image, and the image after binarization is expanded and eroded. In this embodiment, the Canny algorithm is used for edge detection, and then the largest rectangle is searched in the image and the outline of the rectangle is obtained.

[0087] Finally, the information bearing medium image is segmented from the image through perspective transformation. In this embodiment, four-point perspective transformation is performed, such as Figure 4b As shown, the first image including the business card is segmented from the acquired image.

[0088] In this embodiment, another target detection method is provided to identify business card images, such as Figure 5 As shown, in an optional embodiment, the target detection and correction of the information carrying medium in the image based on the acquired image to obtain the first image includes: image binarization based on the acquired image; straight line detection to obtain horizontal straight line set and vertical straight line set grouping, and merge approximately parallel lines to determine the optimal boundary and vertex of the information carrying medium; and segmenting the first image from the image through perspective transformation.

[0089] Specifically, Figure 6a As shown, the image including the deformed business card is distorted:

[0090] First, the image is binarized based on the acquired image.

[0091] Then, straight line detection is performed to obtain horizontal straight line sets and vertical straight line set groups, and approximate parallel lines are merged to determine the optimal boundaries and vertices of the information carrying medium. In this embodiment, LSD straight line detection is used to obtain horizontal straight line sets and vertical straight line set groups in the image, and approximate parallel lines in the horizontal straight line sets and vertical straight line sets are merged to determine the optimal boundaries and vertices in the image.

[0092] Finally, the information bearing medium image is segmented from the image through perspective transformation. In this embodiment, four-point perspective transformation is performed, such as Figure 6b As shown, the first image including the business card is segmented from the acquired image.

[0093] The above-mentioned detection of edges in an image by an edge algorithm, or detection of an image in an image by a line detection algorithm, may result in business card detection failure due to unclear edges of the business card or occlusion of the business card in the image. In this embodiment, an image quality requirement is preset, and the image quality of the first image obtained by edge detection or line detection is judged according to the preset requirement. If the quality requirement is met, text detection is continued, otherwise, the text direction detection is performed on the first image obtained to correct the first image.

[0094] In an optional embodiment, performing text direction detection on the first image and rotating the first image to correct the first image includes: using a preset full-angle text detection and classification model to perform text direction detection on the first image and output a rotation angle; rotating the first image by the rotation angle.

[0095] In this embodiment, a preset full-angle text detection classification model is used, which is based on the VGG16 model and can quickly correct the angle of the image after pre-training and testing, wherein the full angle includes 0-360 degrees, and the acquired image is discretized in steps of 10 degrees to detect the text in the image. The first image is input into the classification model, and the first image is corrected according to the angle output by the classification model.

[0096] Second, performing text recognition on the first image to obtain text content of the information bearing medium.

[0097] Considering that the first image is segmented based on the acquired image, it is inevitable that the text in the image is tilted, distorted, blurred, etc. To solve the problem that the text in the first image is tilted, in an optional embodiment, as Figure 7 As shown, performing text recognition on the first image to obtain the text content of the information-bearing medium includes: performing text detection on the first image to obtain multiple second images; and using a preset text recognition network model to recognize the multiple second images to obtain the text content of the information-bearing medium.

[0098] That is, the text in the first image is corrected by text detection.

[0099] Specifically, Figure 8 As shown, the text detection on the first image to obtain multiple second images includes: obtaining an axisymmetric bounding box based on the first image; performing text score prediction on each of the axisymmetric bounding boxes to obtain a tilted minimum area box; detecting weak text in the tilted minimum area box and obtaining a second image.

[0100] In this embodiment, text is detected in the first image to include text box images of text, that is, the first image is further processed to narrow the image range to text boxes one by one, thereby improving the accuracy and generalization of image processing.

[0101] like Figure 9a-9c As shown, in this embodiment, a text detection network is used to perform text detection, and the text detection network is based on the Faster R-CNN target detection framework.

[0102] First, an oriented axisymmetric bounding box surrounding the text is generated through RPN.

[0103] Then, for each text box, the text score, axisymmetric bounding box and tilted minimum area box are predicted simultaneously.

[0104] Finally, considering that there may be weak and small text in the first image, a small anchor is added to the text detection network to detect weak and small text, and the tilted NMS post-processing detection frame is used to obtain the final detection result.

[0105] The text detection network used in this embodiment can solve the problem of text distortion in images and effectively improve the detection accuracy and speed of text.

[0106] After the first image is subjected to text detection to obtain multiple second images, such as Figure 9a-9c As shown, the second images are of different sizes, and the text in each second image needs to be further recognized through a text recognition network.

[0107] Specifically, the plurality of second images are recognized using a preset text recognition network model to obtain text content of the information bearing medium.

[0108] In this embodiment, the text recognition network model is a CRNN text recognition network model, which combines the characteristics of the CNN network and the RNN network, and can scale the text box image to a fixed length in the vertical direction. Compared with the traditional text recognition model, it no longer performs character segmentation and horizontal scaling, which can improve the recognition speed; at the same time, the model can also recognize character sequences of any length and is no longer limited by the length of the characters. Moreover, compared with the traditional non-end-to-end text recognition network model, the model can achieve end-to-end training, which can improve the accuracy of text recognition. It is worth noting that the training of the model is flexible, and it can train a dictionary-based text recognition network model or a non-dictionary-based text recognition network model, and the model has the characteristics of small size and fast recognition speed.

[0109] The multiple second images obtained by text detection are input into the CRNN text recognition network model, for example, the recognition results are obtained according to the process of CNN (convolutional neural network)-LSTM (long short-term memory network)-CTC (connectivity mechanism time classification), and the second images are recognized as strings, that is, the text content corresponding to each second image is obtained to obtain the text content of the information bearing medium. The CRNN text recognition network model has a high recognition accuracy rate for Chinese and English, and can recognize continuous text of any length.

[0110] At this point, the basic processing of the acquired image including the business card is completed. It is worth noting that other types of information bearing media can also implement preprocessing of the acquired image according to the above steps to prepare for subsequent type classification.

[0111] Third, classify the information bearing medium to obtain the type of the information bearing medium.

[0112] In order to realize textual archiving of the text content of the above-mentioned information-bearing medium, it is necessary to determine the type of the information-bearing medium.

[0113] like Fig.10 As shown, in an optional embodiment, the classifying the information-bearing medium to obtain the type of the information-bearing medium includes: roughly classifying the information-bearing medium according to the aspect ratio of the first image; and finely classifying the roughly classified information-bearing medium based on the first image using a preset corresponding image classifier to obtain the type of the information-bearing medium.

[0114] First, the information bearing medium is roughly classified according to the aspect ratio of the first image.

[0115] Since various information bearing media have different aspect ratios, in this embodiment, the information bearing medium images with quadrilateral outlines segmented from the first image are tiled, and the information bearing media are roughly classified according to a preset aspect ratio range.

[0116] Secondly, the roughly classified information bearing medium is finely classified based on the first image using a preset corresponding image classifier to obtain the type of the information bearing medium.

[0117] In this embodiment, an image classifier corresponding to the aspect ratio of the first image is selected from a plurality of preset image classifiers, and the first image is classified using the selected image classifier to obtain the type of the information bearing medium.

[0118] It is worth noting that the multiple preset image classifiers include image classifiers for various aspect ratios of known information-bearing media, and each of the image classifiers is trained by collecting a large number of samples of various information-bearing media, for example, a large number of taxi invoices, shopping receipts, ordinary invoices, identity cards, passports and other information-bearing media are uniformly scaled into images of the same resolution and labeled with categories; using a commonly used network for classification tasks, a large number of collected samples are input into networks such as inception, vgg, resnet, etc. to train image classifiers corresponding to each of the aspect ratios.

[0119] Since different information-bearing media have different layout rules, background textures, colors, and photos of people, etc., effective classification features can be better learned through the classification task network. In this embodiment, the classification strategy of the information-bearing medium is as follows: judge the shape, that is, classify according to the aspect ratio, and use the image classifier corresponding to each classification to classify the specific information-bearing medium image, so as to obtain the type of the information-bearing medium. It is worth noting that when classifying according to the shape, it is not limited to one classification network model corresponding to each information-bearing medium. Information-bearing media with similar aspect ratios can be classified into one category, such as taxi invoices and shopping receipts are classified into one category, and value-added tax invoices, passports, ID cards, train tickets and waybills are classified into one category.

[0120] like Fig.11 As shown, in an optional embodiment, the classifying the information bearing medium to obtain the type of the information bearing medium includes: roughly classifying the information bearing medium according to the aspect ratio of the first image; and finely classifying the roughly classified information bearing medium based on the text content using a preset corresponding text classifier to obtain the type of the information bearing medium.

[0121] In this embodiment, the specific steps are as follows:

[0122] First, the information bearing medium is roughly classified according to the aspect ratio of the first image.

[0123] Since various information bearing media have different aspect ratios, in this embodiment, the information bearing medium images with quadrilateral outlines segmented from the first image are tiled, and the information bearing media are roughly classified according to a preset aspect ratio range.

[0124] Secondly, the roughly classified information bearing medium is finely classified based on the text content using a preset corresponding text classifier to obtain the type of the information bearing medium.

[0125] In this embodiment, a text classifier corresponding to the aspect ratio of the first image is selected from a plurality of preset text classifiers, and the text classifier is used to perform classification according to the text content to obtain the type of the information bearing medium.

[0126] It is worth noting that the multiple preset text classifiers include text classifiers of various aspect ratios of known information bearing media. In this embodiment, the text classifier is a classification model including LSTM and a fully connected layer, and the training of the text classifier includes: reading data, that is, using Jieba to divide the first 100 characters of the text content obtained by text recognition into words, and marking them according to the type of bill; then, establishing a dictionary and training the text classification model.

[0127] Fourth, the text content is archived according to the type.

[0128] Considering that the same type of information bearing medium also has different layouts, we still take business cards as an example. For example, business cards include various layouts and layouts. The information contained in each business card and its distribution are different. Therefore, different text filing templates need to be set in advance to extract key information from the business card and exclude redundant information. Therefore, the text content obtained by text recognition is filed using the text filing template.

[0129] In an optional embodiment, archiving the text content according to the type includes: archiving the text content according to the type using a preset corresponding text archiving template and obtaining archiving information of the information bearing medium.

[0130] In a specific example, a text filing template is established for business cards, and the template includes 6 types of entries, such as: "name", "position", "company", "address", "mobile phone" and "email", among which "name" is judged to have only 2 to 4 Chinese characters and is unique; "position", "company" and "address" are judged based on keywords, such as "city" and "district" and other keywords are judged as addresses; "mobile phone" is judged to contain 11 consecutive digits or more in the entry, and the first 11 to 10 digits are "13", "15" or "18"; "email" is judged to contain "@" and "com" or "cn" in the entry.

[0131] like Figure 12a-12b The result of text archiving for different business cards is shown in the figure. It can be seen from the figure that, according to the text archiving template, the text of business cards with different layouts can be archived, and the archiving information of the business cards can be extracted according to keywords. The time consumed for archiving business cards with different layouts is similar. At this point, the identification of the information bearing medium is completed, and the identification method is exited.

[0132] Considering that the information-carrying medium may be a label or other form of medium containing a barcode, in an optional embodiment, after acquiring the first image of the information-carrying medium and before performing text recognition on the first image to obtain the text content of the information-carrying medium, the method also includes: performing barcode detection on the first image.

[0133] Taking into account the characteristics of barcodes, in an optional embodiment, the barcode detection on the first image includes: performing image rotation correction on the first image; performing noise reduction processing on the corrected first image; performing edge detection on the noise reduced first image to obtain an edge image; performing line segment detection and filtering on the edge image using probability transformation to determine whether a barcode exists, if not, exiting the barcode detection, and if so, horizontally expanding the line segments at the edge of the barcode to obtain a connected domain, determining the barcode area of ​​the barcode based on the connected domain, and decoding to identify the information bearing medium.

[0134] like Figures 13a-13c As shown, the specific steps of barcode detection and recognition are as follows:

[0135] The first step is to perform image rotation correction on the first image.

[0136] In this embodiment, if Fig.13a As shown, the first image is rotated using the rotation angle obtained by text detection to obtain a forward barcode image.

[0137] The second step is to perform noise reduction on the corrected first image.

[0138] In this embodiment, Gaussian smoothing is performed on the first image to reduce image noise.

[0139] The third step is to perform edge detection on the first image after denoising to obtain an edge image.

[0140] In this embodiment, Canny edge detection is performed on the first image after noise reduction to obtain an edge image.

[0141] The fourth step is to use probability transformation to perform line segment detection and filtering on the edge image to determine whether there is a barcode. If not, exit the barcode detection. If so, horizontally expand the line segments at the edge of the barcode to obtain a connected domain, determine the barcode area of ​​the barcode based on the connected domain, and decode to identify the information bearing medium.

[0142] In this embodiment, the edge image is subjected to probabilistic Hough transform line segment detection, for example, to detect whether the edge image contains the edge of surrounding text or patterns, and to exclude invalid patterns included in the edge image according to the straight line characteristics of the barcode. That is, the line segments obtained by the Hough transform are filtered and sorted, and the line segments that are not the barcode edges are excluded. Specifically, the text in the edge image or the straight line in the border is detected using straight line detection, and the barcode direction information known by the barcode is used to judge the detected straight line to determine whether the detected straight line originates from the barcode. Among them, the filtering of the straight line specifically includes:

[0143] According to the direction of the barcode, since the direction of the barcode is known, if the direction of the line segment and the detected barcode direction exceed the preset threshold, it is determined that there is no barcode in the edge image, the barcode processing is terminated, and the subsequent process is continued to identify the information bearing medium, otherwise the detection is continued.

[0144] If there is a barcode, based on the feature of the barcode edge having a corner point, detect whether the line segment that meets the threshold has at least one end with a corner point feature. If so, the line segment is determined to be a seed line segment, otherwise the line segment is used as a candidate line segment.

[0145] It is determined whether the distance between one endpoint of the candidate line segment and one endpoint of the seed line segment is less than a preset distance threshold, for example, the distance threshold is 2 to 3 pixels. If the distance threshold is satisfied, the candidate line segment is the seed line segment. Fig.13b As shown, the line segment is expanded to obtain a seed line segment.

[0146] Repeat the above process to filter the detected line segments and retain all seed line segments as barcode edges.

[0147] The detected line segments at the edge of the barcode are horizontally expanded to obtain a connected domain, that is, a horizontal kernel is used to expand the parallel edge line segments to obtain a connected region.

[0148] A barcode region of the barcode is determined according to the connected domain, and the barcode is decoded to identify the barcode.

[0149] In this embodiment, the minimum circumscribed rectangle of the connected area is calculated and the rectangular area is used as the barcode area. Fig.13c As shown, the non-black area in the figure is the connected domain obtained after horizontal expansion, that is, the barcode area in the edge image.

[0150] According to the barcode area, the corresponding barcode image is transmitted to the barcode decoding program, for example, the barcode is decoded by the Zbar decoding program, and the specific information of the information bearing medium is obtained after decoding, the identification of the information bearing medium is completed, and the identification method is exited.

[0151] Corresponding to the information-carrying medium identification method provided in the above-mentioned embodiments, an embodiment of the present application also provides an information-carrying medium identification device. Since the information-carrying medium identification device provided in the embodiment of the present application corresponds to the identification methods provided in the above-mentioned embodiments, the aforementioned implementation manner is also applicable to the information-carrying medium identification device provided in this embodiment and will not be described in detail in this embodiment.

[0152] like Fig.14 As shown, an embodiment of the present application also provides an information-bearing medium identification device, including a distortion correction module for acquiring a first image of an information-bearing medium; a text recognition module for performing text recognition on the first image to obtain text content of the information-bearing medium; an information-bearing medium classification module for classifying the information-bearing medium to obtain the type of the information-bearing medium; and a text archiving module for archiving the text content according to the type.

[0153] In an optional embodiment, the information bearing medium identification device further includes: a barcode detection module, configured to perform barcode detection on the first image.

[0154] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements: based on the acquired image, performing distortion correction on the information-carrying medium in the image to obtain an information-carrying medium image; performing text detection on the information-carrying medium image to obtain a plurality of text frame images; performing text recognition on each of the text frame images to obtain a plurality of text contents; classifying the information-carrying medium to obtain a type of the information-carrying medium; and text-archiving the plurality of text contents according to the type.

[0155] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive lists) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this real-time example, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0156] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0157] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0158] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0159] like Fig.15 As shown, a structural schematic diagram of a computer device provided by another embodiment of the present invention. Fig.15 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0160] like Fig.15 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0161] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0162] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0163] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.15 not shown, usually called a "hard drive"). Although Fig.15 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0164] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0165] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Fig.15 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Fig.15 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0166] The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a method for identifying an information bearing medium provided in an embodiment of the present invention.

[0167] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A method for identifying an information bearing medium, characterized in that: include: Acquiring a first image of an information-bearing medium, comprising: performing target detection and correction on the information-bearing medium in the image based on the acquired image to acquire the first image; determining whether the first image meets preset requirements, and if not, performing text direction detection and rotation on the first image to correct the first image, comprising: performing text direction detection on the first image using a preset full-angle text detection classification model and outputting a rotation angle, rotating the first image by the rotation angle, wherein the full angle includes 0-360 degrees, and discretizing the acquired image in steps of 10 degrees to detect text in the image; Performing text recognition on the first image to obtain text content of the information carrying medium includes: performing text detection on the first image to obtain multiple second images, where the second images are text boxes, including: obtaining an axisymmetric bounding box based on the first image; performing text score prediction on each of the axisymmetric bounding boxes to obtain a tilted minimum area box; detecting weak text in the tilted minimum area box and obtaining a second image; using a preset text recognition network model to recognize the multiple second images to obtain the text content of the information carrying medium; Classifying the information bearing medium to obtain the type of the information bearing medium includes: roughly classifying the information bearing medium according to the aspect ratio of the first image; finely classifying the roughly classified information bearing medium based on the first image using a preset corresponding image classifier to obtain the type of the information bearing medium, or finely classifying the roughly classified information bearing medium based on the text content using a preset corresponding text classifier to obtain the type of the information bearing medium; The text content is filed according to the type.

2. The identification method according to claim 1, characterized in that: The performing target detection and correction on the information bearing medium in the image based on the acquired image to acquire the first image comprises: Perform image binarization based on the acquired image; Performing edge detection to obtain the outline of the largest rectangle in the image; The first image is segmented from the image by perspective transformation.

3. The identification method according to claim 1, characterized in that: The performing target detection and correction on the information bearing medium in the image based on the acquired image to acquire the first image comprises: Perform image binarization based on the acquired image; Performing straight line detection to obtain horizontal straight line sets and vertical straight line sets grouping, and merging approximately parallel lines to determine the optimal boundary and vertex of the information bearing medium; The first image is segmented from the image by perspective transformation.

4. The identification method according to claim 1, characterized in that: The textual archiving of the text content according to the type includes: According to the type, the text content is archived using a preset corresponding text archiving template and the archiving information of the information bearing medium is obtained.

5. The identification method according to claim 1, characterized in that: After acquiring the first image of the information carrying medium and before performing text recognition on the first image to acquire text content of the information carrying medium, the method further includes: Perform barcode detection on the first image.

6. The identification method according to claim 1, characterized in that: The performing barcode detection on the first image comprises: performing image rotation correction on the first image; Performing noise reduction processing on the corrected first image; Performing edge detection on the first image after noise reduction to obtain an edge image; Probabilistic transformation is used to perform line segment detection and filtering on the edge image to determine whether a barcode exists. If not, the barcode detection is exited. If yes, the line segment at the edge of the barcode is horizontally expanded to obtain a connected domain, the barcode area of ​​the barcode is determined based on the connected domain, and decoding is performed to identify the information bearing medium.

7. An information bearing medium identification device, characterized in that: include A distortion correction module, used for acquiring a first image of an information bearing medium, comprising: performing target detection and correction on the information bearing medium in the image based on the acquired image to acquire the first image; determining whether the first image meets preset requirements, and if not, performing text direction detection and rotation on the first image to correct the first image, comprising: performing text direction detection on the first image using a preset full-angle text detection classification model and outputting a rotation angle, rotating the first image by the rotation angle, wherein the full angle includes 0-360 degrees, and discretizing the acquired image in steps of 10 degrees to detect text in the image; A text recognition module, used for performing text recognition on the first image to obtain text content of the information carrying medium, including: performing text detection on the first image to obtain multiple second images, wherein the second images are text boxes, including: obtaining an axisymmetric bounding box according to the first image; performing text score prediction on each of the axisymmetric bounding boxes to obtain a tilted minimum area box; detecting weak text in the tilted minimum area box and obtaining a second image; using a preset text recognition network model to recognize the multiple second images to obtain the text content of the information carrying medium; An information bearing medium classification module, used for classifying the information bearing medium to obtain the type of the information bearing medium, including: being used for roughly classifying the information bearing medium according to the aspect ratio of the first image; being used for finely classifying the roughly classified information bearing medium based on the first image using a preset corresponding image classifier to obtain the type of the information bearing medium, or being used for finely classifying based on the text content using a preset corresponding text classifier to obtain the type of the information bearing medium; The text archiving module is used to archive the text content according to the type.

8. The information bearing medium identification device according to claim 7, characterized in that: Also includes: A barcode detection module is used to perform barcode detection on the first image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the information bearing medium identification method as described in any one of claims 1 to 6 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images