A text calibration method, device, equipment and storage medium

Through text calibration methods and devices, the text to be calibrated in the enterprise qualification certificate image is calibrated using the reference text in the reference library, which solves the problem that image quality affects the OCR recognition results, and improves the accuracy and efficiency of document information entry and review.

CN114092684BActive Publication Date: 2024-11-19CHINA UNIONPAY
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202111360903.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-17
Publication Date
2024-11-19
Estimated Expiration
2041-11-17

AI Technical Summary

Technical Problem

In business scenarios such as merchant networking material review, financial product account opening and enterprise bidding, image quality problems of enterprise qualification certificates lead to incorrect OCR technology identification results, affecting the efficiency of document information entry and review.

Method used

Provide a text calibration method and device, by acquiring the target text content in the document image, obtaining reference text from the reference library based on the unique identification. If the similarity is greater than the preset threshold, the reference text is used as the calibration result to effectively calibrate the waiting time. Text, identify and correct identification errors, identify forged text.

Benefits of technology

It improves the accuracy and efficiency of document information entry and review, effectively identify and correct identification errors, and prevents misidentification of forged texts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114092684B_ABST
    Figure CN114092684B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a text calibration method, device, equipment and storage medium, which relate to the field of artificial intelligence technology. The method includes: obtaining an image to be processed of a target object, identifying the target text content contained in the image to be processed, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated. Based on the unique identifier of the target object, obtain a reference text corresponding to each of the at least one text to be calibrated from a reference library. If the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as a calibration result corresponding to at least one text to be calibrated. Therefore, in the present application, the text to be calibrated is calibrated according to the similarity between the text to be calibrated and the corresponding reference text. In the case where there are partial recognition errors in the text to be calibrated, the text to be calibrated can be effectively calibrated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a text calibration method, device, equipment and storage medium. Background Art

[0002] In business scenarios such as merchant network access material review, financial product account opening, and corporate bidding, it is necessary to enter the enterprise qualification certificates into the system and review the certificate information. Since the information contained in the enterprise qualification certificates is relatively complex, OCR (Optical Character Recognition) technology is generally used to assist in the entry of certificate information, which can effectively improve the efficiency of certificate information entry.

[0003] When using OCR technology to input certificate information, the enterprise qualification certificate is photographed as a certificate image, and the certificate image is subjected to OCR recognition. If there is an image quality problem with the certificate image, it will seriously affect the recognition result of the OCR technology, resulting in many errors in the certificate information entered into the system, increasing the workload of subsequent certificate information review, and seriously affecting the efficiency of certificate information entry and review. Summary of the invention

[0004] The embodiments of the present application provide a text calibration method, apparatus, device and storage medium for improving the accuracy of text recognition, thereby improving the efficiency of information entry and review.

[0005] On the one hand, an embodiment of the present application provides a text calibration method, the method comprising:

[0006] Obtain an image to be processed of a target object;

[0007] Identifying target text content contained in the image to be processed, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated;

[0008] Based on the unique identifier of the target object, obtaining reference texts corresponding to the at least one text to be calibrated from a reference library;

[0009] If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as the calibration result corresponding to the at least one text to be calibrated.

[0010] In one aspect, an embodiment of the present application provides a text calibration device, the device comprising:

[0011] An acquisition module, used for acquiring an image to be processed of a target object;

[0012] A recognition module, used for recognizing target text content contained in the image to be processed, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated;

[0013] A retrieval module, configured to obtain, from a reference library, reference texts corresponding to the at least one text to be calibrated based on the unique identifier of the target object;

[0014] The comparison module is used to use each reference text obtained as a calibration result corresponding to the at least one text to be calibrated if the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold.

[0015] Optionally, the identification module is specifically used for:

[0016] Identifying at least one target text detection frame from the image to be processed, and respectively identifying subtext content contained in the at least one target text detection frame;

[0017] The target text content is obtained based on the identified sub-text contents.

[0018] Optionally, the identification module is further used for:

[0019] Performing feature extraction on the image to be processed to obtain at least two image features of different sizes;

[0020] fusing the at least two image features of different sizes to obtain the target image feature;

[0021] Based on the target image feature, determining at least one first candidate text detection frame and a confidence level corresponding to each of the at least one first candidate text detection frame;

[0022] Based on the obtained confidence levels, the at least one first candidate text detection frame is screened to obtain at least one target text detection frame.

[0023] Optionally, the identification module is further used for:

[0024] Iteratively fusing the at least two image features of different sizes until the at least two image features of different sizes are fused into one fused feature, obtaining a target fused feature, and performing feature extraction on the target fused feature to obtain the target image feature;

[0025] Each iterative fusion includes the following steps:

[0026] Selecting the image feature with the smallest size from among the image features retained after the previous iterative fusion as the reference feature;

[0027] Based on the size of the reference feature, the fusion feature obtained by the previous iterative fusion is adjusted, and the fusion feature obtained after the size adjustment is fused with the reference feature to obtain the fusion feature of this iterative fusion.

[0028] Optionally, the identification module is further used for:

[0029] Acquire, from the at least one first candidate text detection frame, a second candidate text detection frame whose confidence is greater than or equal to a confidence threshold;

[0030] Based on the obtained overlapping properties between the second candidate text detection frames, the at least one target text detection frame is obtained from the second candidate text detection frames.

[0031] Optionally, the identification module is further used for:

[0032] Sorting the second candidate text detection boxes in descending order of confidence to obtain a sorting result;

[0033] According to the sorting result, a reference detection frame is selected from each of the retained second candidate text detection frames in turn, and based on the overlapping properties between the selected reference detection frame and the other retained second candidate text detection frames, the other retained second candidate text detection frames are screened to obtain the at least one target text detection frame.

[0034] Optionally, the identification module is further used for:

[0035] For the second candidate text detection box ranked at the Nth position in the sorting result, the following screening steps are performed:

[0036] Determine the intersection-and-union ratios between the Nth second candidate text detection frame and other retained second candidate text detection frames respectively; wherein N is an integer greater than 0;

[0037] The second candidate text detection frames whose intersection-over-union ratio is greater than a preset threshold value are deleted from the other retained second candidate text detection frames.

[0038] Optionally, the identification module is further used for:

[0039] For the at least one target text detection frame, perform the following operations respectively:

[0040] Extract features of an image within a target text detection frame to obtain detection image features;

[0041] Based on the detected image features, determining the temporal features corresponding to the image within the target text detection frame;

[0042] The temporal features are identified to obtain the sub-text content contained in the target text detection frame.

[0043] Optionally, the at least one text to be calibrated includes a first text to be calibrated, and the first text to be calibrated corresponds to a first reference text;

[0044] The system also includes a similarity calculation module, which is specifically used to:

[0045] If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as the calibration result corresponding to the at least one text to be calibrated,

[0046] determining a first text overlap between the first to-be-calibrated text and the first reference text;

[0047] Determining a first overlapping mark based on the correlation between the first text overlapping degree and the overlapping degree threshold;

[0048] A first similarity between the first text to be calibrated and the first reference text is determined according to the first text overlap degree, the first overlap mark, and the text lengths corresponding to the first text to be calibrated and the first reference text.

[0049] Optionally, the similarity calculation module is further used for:

[0050] Determining a feature vector to be calibrated of the first text to be calibrated based on the number of occurrences of each character contained in the first text to be calibrated in the text dictionary;

[0051] Determining a reference feature vector of the first reference text based on the number of occurrences of each word contained in the first reference text in a text dictionary;

[0052] Compare the eigenvalues ​​to be calibrated of each dimension in the eigenvector to be calibrated with the reference eigenvalues ​​of the corresponding dimensions in the reference eigenvector, respectively, to obtain the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector;

[0053] Based on the total overlap dimension, a first text overlap degree between the first to-be-calibrated text and the first reference text is determined.

[0054] Optionally, the similarity calculation module is further used for:

[0055] For the eigenvalues ​​to be calibrated in each dimension, the following steps are performed respectively: comparing one eigenvalue to be calibrated with a corresponding reference eigenvalue, and based on the comparison result, obtaining a minimum eigenvalue from the one eigenvalue to be calibrated and the corresponding reference eigenvalue;

[0056] Based on the obtained minimum eigenvalues ​​of the various dimensions, the total overlapping dimension between the feature vector to be calibrated and the reference feature vector is determined.

[0057] Optionally, the at least one text to be calibrated further includes a second text to be calibrated, and the second text to be calibrated corresponds to the second reference text;

[0058] The similarity calculation module is also used for:

[0059] Comparing the text lengths corresponding to the second text to be calibrated and the second reference text, and determining the shortest text length and the second overlap mark based on the comparison result;

[0060] comparing the second text to be calibrated with the second reference text to determine at least one overlapping word between the second text to be calibrated and the second reference text;

[0061] Determine a second text overlap degree based on the positions of the at least one overlapping character in the second text to be calibrated and the second reference text respectively;

[0062] Based on the second overlap mark, the second text overlap degree, and the shortest text length, a second similarity between the second text to be calibrated and the second reference text is determined.

[0063] Optionally, the similarity calculation module is further used for:

[0064] If the text length of the second text to be calibrated is the same as the text length of the second reference text, the second overlap flag is determined to be 1, and the text length of the second text to be calibrated or the second reference text is used as the shortest text length;

[0065] If the text length of the second text to be calibrated is greater than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second reference text is used as the shortest text length;

[0066] If the text length of the second text to be calibrated is less than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second text to be calibrated is used as the shortest text length.

[0067] Optionally, the second text overlap includes a first sub-overlap and a second sub-overlap;

[0068] The similarity calculation module is also used for:

[0069] For the at least one overlapping character, the following steps are respectively performed: if the position of an overlapping character in the second to-be-calibrated text and the second reference text is the same, the overlapping character is used as a forward overlapping character, otherwise the overlapping character is used as a reverse overlapping character;

[0070] The number of positive overlapping characters in the at least one overlapping character is used as a first sub-overlapping degree;

[0071] The number of reverse overlapping characters in the at least one overlapping character is used as the second sub-overlapping degree.

[0072] Optionally, the at least one text to be calibrated includes: a merchant name to be calibrated and a legal person name to be calibrated, and the reference text corresponding to each of the at least one text to be calibrated includes: a reference merchant name and a reference legal person name.

[0073] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned text calibration method when executing the program.

[0074] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device, and when the program is run on the computer device, the computer device executes the steps of the above-mentioned text calibration method.

[0075] In an embodiment of the present application, an image to be processed of a target object is obtained, and target text content contained in the image to be processed is identified, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated. Based on the unique identifier of the target object, a reference text corresponding to each of the at least one text to be calibrated is obtained from a reference library. If the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as a calibration result corresponding to at least one text to be calibrated. Because, in the present application, the text to be calibrated is calibrated according to the similarity between the text to be calibrated and the corresponding reference text. In the case where there are partial recognition errors in the text to be calibrated, the text to be calibrated can be effectively calibrated. At the same time, if the text content in the image to be processed is forged, the similarity between the text to be calibrated and the corresponding reference text is low, and the forged text content of the image to be processed can be effectively identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0077] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0078] Figure 2 A flowchart of a text calibration method provided in an embodiment of the present application;

[0079] Figure 3 A schematic diagram of the structure of an image to be processed provided in an embodiment of the present application;

[0080] Figure 4 A schematic diagram of the structure of an image to be processed provided in an embodiment of the present application;

[0081] Figure 5 A schematic diagram of the structure of a target detection model provided in an embodiment of the present application;

[0082] Figure 6 A schematic diagram of a process for identifying a target text detection frame provided in an embodiment of the present application;

[0083] Figure 7 A schematic diagram of the structure of a first candidate text detection frame provided in an embodiment of the present application;

[0084] Figure 8 A schematic diagram of the structure of a rectangular detection frame provided in an embodiment of the present application;

[0085] Fig. 9 A schematic diagram of the structure of a first candidate text detection frame provided in an embodiment of the present application;

[0086] Fig.10 A schematic diagram of the structure of a first candidate text detection frame provided in an embodiment of the present application;

[0087] Fig.11 A structural diagram of a text recognition model provided in an embodiment of the present application;

[0088] Fig.12 A schematic diagram of a process for determining a first text overlap degree provided in an embodiment of the present application;

[0089] Fig.13 A schematic diagram of a process for determining a second similarity provided in an embodiment of the present application;

[0090] Fig.14A flowchart of a text calibration method provided in an embodiment of the present application;

[0091] Fig.15 A structural schematic diagram of a text calibration device provided in an embodiment of the present application;

[0092] Fig.16 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0093] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0094] refer to Figure 1 , which is a system architecture diagram applicable to an embodiment of the present application, and the system architecture at least includes a terminal device 101 and a text calibration system 102.

[0095] The terminal device 101 is installed with a target application for text calibration, which may be a pre-installed client, a web application, or a small program embedded in other applications, etc. The terminal device 101 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0096] The text calibration system 102 is a backend server of the target application, providing services for the target application. The text calibration system 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0097] The terminal device 101 and the text calibration system 102 may be connected directly or indirectly via wired or wireless communication, which is not limited in the present application.

[0098] The terminal device 101 responds to the user's text calibration operation and sends a text calibration instruction to the text calibration system 102. The text calibration system 102 receives the text calibration instruction and obtains the image to be processed of the target object, wherein the target object can be a certificate, etc., which is not limited here. The text calibration system 102 identifies the target text content contained in the image to be processed, and the target text content includes the unique identifier of the target object and at least one text to be calibrated. Based on the unique identifier of the target object, the reference text corresponding to each of the at least one text to be calibrated is obtained from the reference library. Finally, if the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as the calibration result corresponding to at least one text to be calibrated.

[0099] based on Figure 1 The system architecture diagram, the present application embodiment provides a process of a text calibration method, such as Figure 2 As shown, the process of this method is Figure 1 The text calibration system 102 shown performs the following steps:

[0100] Step S201, obtaining an image to be processed of a target object.

[0101] Specifically, the target object can be an enterprise qualification certificate or other certificates.

[0102] The text calibration system includes an image acquisition device for acquiring image information of the target object. The image acquisition device can be a camera, a camcorder, a camera, a scanner, or other devices with a camera function, such as a mobile phone, a tablet computer, etc.

[0103] Due to problems with the acquisition equipment and environment, the images to be processed are prone to problems such as low image brightness, low contrast, uneven lighting, and perspective deformation. At the same time, there may also be problems such as text tilt and text bending in the image, which will reduce the accuracy of text recognition.

[0104] Step S202 : identifying target text content contained in the image to be processed, wherein the target text content includes a unique identifier of a target object and at least one text to be calibrated.

[0105] Specifically, the unique identifier is used to distinguish the target object from other target objects, and the unique identifier may be a unified social credit code, an ID number, a telephone number, etc. The at least one text to be calibrated includes: a merchant name to be calibrated and a legal person name to be calibrated.

[0106] Since the image to be processed may have the above-mentioned image problems, when the image to be processed is recognized, there may be a deviation between the obtained target text content and the text in the image.

[0107] For example, Figure 3 The unified social credit code recorded in the enterprise qualification certificate image shown is "123456ABC", the name of the merchant to be calibrated is "Gansu Wuhu Simei Trading Co., Ltd.", and the name of the legal person to be calibrated is "Zhang San".

[0108] Image recognition is performed on the enterprise qualification certificate image, and the unified social credit code is obtained as "123456ABC", the merchant name to be calibrated is "Gansu Wuhu Simei Trading Co., Ltd.", and the legal person name to be calibrated is "Zhang Er". Due to the image quality problem of the enterprise qualification certificate image, the merchant name to be calibrated and the legal person name to be calibrated are incorrect, so they need to be calibrated.

[0109] Step S203: based on the unique identifier of the target object, obtaining from the reference library at least one reference text corresponding to each of the texts to be calibrated.

[0110] Specifically, if the unique identifier obtained from the image to be processed is correct, at least one reference text corresponding to each of the texts to be calibrated is obtained from the reference library; otherwise, it is determined that the unique identifier does not exist in the reference library and no processing is performed.

[0111] For example, Figure 3 After identification, based on the obtained unified social credit code "123456ABC", the reference text is obtained from the benchmark database, that is, the reference merchant name is "Gansu Wuhu Sihai Trading Co., Ltd." and the reference legal person name is "Zhang San".

[0112] Step S204: if the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as a calibration result corresponding to the at least one text to be calibrated.

[0113] When the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, it indicates that the recognition result of the image to be processed is relatively accurate. Therefore, the reference text can be directly used as the calibration result of the text to be calibrated.

[0114] When the similarity between at least one text to be calibrated and the corresponding reference text is not greater than a preset threshold, it indicates that the recognition result of the image to be processed is inaccurate, or the image to be processed is forged. Therefore, the reference text is not used directly for calibration, but the image to be processed is annotated for subsequent verification.

[0115] Optionally, a preset threshold may be set for all texts to be calibrated, and different preset thresholds may be set for different texts to be calibrated.

[0116] For example, if Figure 3As shown, image recognition is performed on the image of the enterprise qualification certificate, and the unified social credit code obtained is "123456ABC", the name of the merchant to be calibrated is "Gansu Wuhu Simei Trading Co., Ltd.", and the name of the legal person to be calibrated is "Zhang Er".

[0117] For the unique identifier "123456ABC", the reference text obtained from the benchmark library, the reference merchant name is "Gansu Wuhu Sihai Trading Co., Ltd.", and the reference legal person name is "Zhang San".

[0118] Compare "Gansu Province Wuhu Simei Trading Co., Ltd." with "Gansu Province Wuhu Sihai Trading Co., Ltd." to obtain the first similarity; compare "Zhang Er" with "Zhang San" to obtain the second similarity. If both the first similarity and the second similarity are greater than the preset threshold, "Gansu Province Wuhu Sihai Trading Co., Ltd." is used as the calibration result of the merchant name to be calibrated, and "Zhang San" is used as the calibration result of the legal person name to be calibrated.

[0119] In an embodiment of the present application, an image to be processed of a target object is obtained, and target text content contained in the image to be processed is identified, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated. Based on the unique identifier of the target object, a reference text corresponding to each of the at least one text to be calibrated is obtained from a reference library. If the similarity between at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as a calibration result corresponding to at least one text to be calibrated. Because, in the present application, the text to be calibrated is calibrated according to the similarity between the text to be calibrated and the corresponding reference text. In the case where there are partial recognition errors in the text to be calibrated, the text to be calibrated can be effectively calibrated. At the same time, if the text content in the image to be processed is forged, the similarity between the text to be calibrated and the corresponding reference text is low, and the forged text content of the image to be processed can be effectively identified.

[0120] Optionally, in the above step S202, identifying the target text content contained in the image to be processed includes the following steps:

[0121] At least one target text detection frame is identified from the image to be processed, sub-text contents contained in the at least one target text detection frame are identified respectively, and target text contents are obtained based on the identified sub-text contents.

[0122] Specifically, for the content that needs to be calibrated later, the target text content is determined from a plurality of sub-text contents.

[0123] For example, if Figure 4As shown, there are 9 target text detection boxes, namely target text detection box 401, target text detection box 402, target text detection box 403, target text detection box 404, target text detection box 405, target text detection box 406, target text detection box 407, target text detection box 408, and target text detection box 409.

[0124] The images in the 9 text detection boxes are recognized to obtain the corresponding sub-text contents, which are "Business License", "Uniform Social Credit Code", "123456ABC", "Name", "Gansu Wuhu Sihai Trading Co., Ltd.", "Residence", "Lanzhou City, Gansu Province", "Legal Representative", and "Zhang San".

[0125] From the 9 sub-text contents, 3 sub-text contents are determined as target text contents, namely "123456ABC", "Gansu Wuhu Sihai Trading Co., Ltd." and "Zhang San".

[0126] Optionally, a target detection algorithm can be used to perform text detection on the processed image to obtain a target text detection frame, wherein commonly used target detection algorithms include traditional feature extraction methods and deep learning detection algorithms. Traditional feature extraction methods include SWT algorithm, MSER algorithm, HoG algorithm, etc. Deep learning detection algorithms include CTPN algorithm, TextBoxes algorithm, EAST algorithm, etc.

[0127] The present application embodiment proposes a target detection model for performing text detection on an image to be processed and identifying at least one target text detection frame. In order to more clearly introduce the target detection model of the present application embodiment, Figure 5 The structure of a target detection model is exemplified. The target detection model consists of a feature extraction module, a feature merging module and a result output module.

[0128] Specifically, identifying at least one target text detection frame from the image to be processed includes the following steps: Figure 6 As shown:

[0129] Step S601: a feature extraction module extracts features from an image to be processed to obtain image features of at least two different sizes.

[0130] In a possible implementation, the image to be processed is directly input into a feature extraction module for feature extraction.

[0131] Another possible implementation is to input the image to be processed into a reference module to obtain image features, and then input the image features into a feature extraction module to extract features. The reference module can be resnet18, PVANet, VGG, etc., which is not limited here.

[0132] Specifically, the feature extraction module includes a plurality of different convolution modules, each of which includes at least one convolution layer, and each of which may further include a downsampling layer.

[0133] The feature extraction module extracts image features of different sizes through each convolution module. The size of the image feature output by the previous convolution module is larger than the size of the image feature output by the next convolution module. The sizes of the image features can be multiples or any other relationship.

[0134] For example, if Figure 5 As shown, the feature extraction module includes 4 convolution modules for obtaining image features of 4 different sizes.

[0135] Input the image to be processed into the reference module to obtain the image feature f, and the size of the image feature f is (w, h). Input the image feature f into the first convolution module C1 to obtain the image feature f4, and the size of the image feature f4 is (w / 4, h / 4). Input the image feature f4 into the second convolution module C2 to obtain the image feature f3, and the size of the image feature f3 is (w / 8, h / 8). Input the image feature f3 into the third convolution module C3 to obtain the image feature f2, and the size of the image feature f2 is (w / 16, h / 16). Input the image feature f2 into the fourth convolution module C4 to obtain the image feature f1, and the size of the image feature f1 is (w / 32, h / 32).

[0136] Step S602: The feature merging module fuses at least two image features of different sizes to obtain target image features.

[0137] Optionally, at least two image features of different sizes are iteratively fused until at least two image features of different sizes are fused into one fused feature, a target fused feature is obtained, and feature extraction is performed on the target fused feature to obtain a target image feature. A convolution module may be used to extract the target fused feature.

[0138] Each iterative fusion includes the following steps:

[0139] If it is the first iterative fusion, the image feature with the second smallest size is selected from at least two image features of different sizes as the benchmark feature, and then the image feature with the smallest size is adjusted based on the size of the benchmark feature, and the image feature obtained after the adjustment is fused with the benchmark feature to obtain the fusion feature of this iterative fusion.

[0140] Otherwise, the image feature with the smallest size among the image features retained after the previous iterative fusion is selected as the reference feature. Then, based on the size of the reference feature, the fusion feature obtained by the previous iterative fusion is adjusted, and the fusion feature obtained after the adjustment is fused with the reference feature to obtain the fusion feature of this iterative fusion.

[0141] Specifically, an upsampling layer may be used to resize the image features. In order to increase the diversity of features, a convolution operation may be performed on the fused features.

[0142] For example, if Figure 5 As shown, the feature merging module obtains four image features of different sizes, namely, image feature f4, corresponding to a size of (w / 4, h / 4); image feature f3, corresponding to a size of (w / 8, h / 8); image feature f2, corresponding to a size of (w / 16, h / 16); image feature f1, corresponding to a size of (w / 32, h / 32).

[0143] When performing the first iterative fusion, the image feature with the smallest size is image feature f1, and the image feature with the second smallest size is image feature f2. Set image feature f1 to image feature h1, and use image feature f2 as the reference feature. Based on the size of image feature f2 (w / 16, h / 16), perform a 2x upsampling operation on image feature h1, and adjust the size of image feature h1 from (w / 32, h / 32) to (w / 16, h / 16). Then fuse image feature f2 and the adjusted image feature h1 to obtain fused feature h2, with a size of (w / 16, h / 16). Then use convolution module C5 to extract features from fused feature h2 to obtain image feature h2', where the size of h2' is (w / 16, h / 16), and convolution module C5 includes 1x1 convolution and 3x3 convolution.

[0144] During the second iterative fusion, the retained image features are image feature f4 and image feature f3. Image feature f3 has the smallest size. Image feature f3 is selected as the reference feature. Based on the size of image feature f3 (w / 8, h / 8), image feature h2' is upsampled by 2 times to adjust the size of image feature h2' to (w / 8, h / 8). Image feature f3 and the adjusted image feature h2' are then fused to obtain fused feature h3, which has a size of (w / 8, h / 8). Convolution module C6 is then used to extract features from fused feature h3 to obtain image feature h3', where the size of h3' is (w / 8, h / 8). Convolution module C6 includes 1x1 convolution and 3x3 convolution.

[0145] During the third iterative fusion, the only image feature retained is image feature f4. Image feature f4 is used as the reference feature. Based on the size of image feature f4 (w / 4, h / 4), image feature h3' is upsampled by 2 times, and the size of image feature h3' is adjusted to (w / 4, h / 4). Image feature f4 and the adjusted image feature h3' are then fused to obtain fused feature h4, with a size of (w / 4, h / 4). Convolution module C7 is then used to extract features from fused feature h4 to obtain image feature h4', where the size of h4' is (w / 4, h / 4). Convolution module C7 includes 1x1 convolution and 3x3 convolution.

[0146] When the iterative fusion stops, the feature h4' is the target fusion feature, and the feature h4' is extracted using the convolution module C8 to obtain the target image feature h5, wherein the convolution module C8 includes a 3x3 convolution.

[0147] In an embodiment of the present application, image features of different sizes are obtained and fused, which effectively solves the problem of inconsistent text content sizes in different images to be processed and increases the accuracy of text detection.

[0148] Step S603: Based on the target image features, the result output module determines at least one first candidate text detection box and a confidence level corresponding to each of the at least one first candidate text detection boxes.

[0149] Specifically, the confidence indicates the possibility of detecting the first candidate text in each first candidate text detection frame. The higher the confidence, the greater the possibility that the first candidate text is contained in the first candidate text detection frame; otherwise, the smaller the possibility that the first candidate text is contained in the first candidate text detection frame.

[0150] In a possible implementation, the result output module outputs the coordinates of four vertices. The coordinates of the four vertices are directly connected in sequence to determine the first candidate text detection frame. The first candidate text detection frame can be a detection frame of any shape.

[0151] For example, the coordinates of the detection box in the result output module are {(30,10), (60,30), (65,45), (30,60)}. The first candidate text detection box is determined, such as Figure 7 shown.

[0152] In the embodiment of the present application, since the result output module can output any four vertex coordinates, the first candidate text detection box can be of any shape, and the position of the candidate text can be detected more accurately, thereby increasing the accuracy of detection.

[0153] In another possible implementation, the result output module outputs the coordinates of the upper left corner vertex, the detection frame width, the detection frame height, and the inclination. A rectangular detection frame is determined based on the coordinates of the upper left corner vertex, the detection frame width, and the detection frame height, and then the rectangular detection frame is adjusted based on the inclination to determine the first candidate text detection frame.

[0154] Specifically, when adjusting the rectangular detection frame, it can be rotated according to a preset rule, for example, the position of the upper left corner vertex is fixed, and the rectangular detection frame is rotated clockwise with a specific inclination; or the position of the lower right corner vertex is fixed, and the rectangular detection frame is rotated counterclockwise with a specific inclination.

[0155] For example, the coordinates of the upper left corner vertex output in the result output module are (30,10), the detection frame width is 30, the detection frame height is 50, and the inclination is 15 degrees. According to the coordinates of the upper left corner vertex and the detection frame width and height, the rectangular detection frame is determined as follows: Figure 8 Then adjust the rectangular detection frame according to the inclination to determine the first candidate text detection frame as shown in Fig. 9 shown.

[0156] In the embodiment of the present application, the first candidate text detection frame is determined by rotating the rectangular detection frame. Therefore, the shape of the first candidate text detection frame is relatively regular, which facilitates obtaining the text content in the first candidate text detection frame in a relatively neat shape.

[0157] Step S604: Based on the obtained confidence levels, the result output module screens at least one first candidate text detection frame to obtain at least one target text detection frame.

[0158] Specifically, a second candidate text detection frame having a confidence greater than or equal to a confidence threshold is obtained from at least one first candidate text detection frame, and then at least one target text detection frame is obtained from each second candidate text detection frame based on the obtained overlapping properties between the second candidate text detection frames.

[0159] Optionally, obtaining at least one target text detection frame from each second candidate text detection frame based on the obtained overlapping attributes between each second candidate text detection frame comprises the following steps:

[0160] The second candidate text detection frames are sorted in descending order of confidence to obtain a sorting result. According to the sorting result, reference detection frames are selected from the retained second candidate text detection frames in turn, and based on the overlapping properties between the selected reference detection frame and the other retained second candidate text detection frames, the other retained second candidate text detection frames are screened to obtain at least one target text detection frame.

[0161] Specifically, the overlapping attribute may be an intersection-over-union ratio between the reference detection frame and the second candidate text detection frame, or may be an intersection area between the reference detection frame and the second candidate text detection frame.

[0162] For the second candidate text detection box ranked Nth in the sorting results, perform the following steps:

[0163] Determine the intersection-and-union ratios between the Nth second candidate text detection frame and other retained second candidate text detection frames, and delete the second candidate text detection frames whose intersection-and-union ratios are greater than a preset threshold among the other retained second candidate text detection frames, where N is an integer greater than 0.

[0164] For example, if Fig.10 As shown, four first candidate target text detection frames are obtained through the target detection model, namely, first candidate target text detection frame 1001, first candidate target text detection frame 1002, first candidate target text detection frame 1003, and first candidate target text detection frame 1004. The confidences corresponding to the four first candidate target text detection frames are 0.9, 0.8, 0.1, and 0.7, respectively.

[0165] The confidence threshold is set to 0.5, the above four first candidate target text detection frames are filtered, the first candidate target text detection frame with a confidence greater than 0.5 is determined as the second candidate text detection frame, and three second candidate text detection frames are obtained, namely, second candidate text detection frame 1001, second candidate target text detection frame 1002, and second candidate target text detection frame 1004.

[0166] The above three second candidate text detection frames are sorted in descending order of confidence, and the sorting results are second candidate text detection frame 1001, second candidate target text detection frame 1002, and second candidate target text detection frame 1004.

[0167] For the above three second candidate text detection frames, the second candidate text detection frame 1001 with the largest confidence is selected as the benchmark detection frame, and then the first intersection and union ratio of the second candidate text detection frame 1001 and the second candidate text detection frame 1002 is determined to be 0.93, and the second intersection and union ratio of the second candidate text detection frame 1001 and the second candidate text detection frame 1004 is determined to be 0.

[0168] The IoU threshold is set to 0.5. Since the first IoU is greater than the IoU threshold, the second candidate text detection frame 1002 is deleted. Since the second IoU is less than the IoU threshold 0.5, the second candidate text detection frame 1004 is retained.

[0169] Detect the reserved second candidate text detection boxes 1001 and 1004. Select the second candidate text detection box 1004 with the second largest confidence as the reference detection box, and then determine that the third intersection over union of the second candidate text detection box 1004 and the second candidate text detection box 1001 is 0. Since the third intersection over union is less than the intersection over union threshold of 0.5, the second candidate text detection box 1001 is retained.

[0170] So far, the judgment is over, and the second candidate text detection box 1001 and the second candidate text detection box 1004 are the target text detection boxes.

[0171] In the embodiment of the present application, based on the overlapping attributes between each second candidate text detection box, the second candidate text detection boxes are filtered, effectively removing the overlapping second candidate text detection boxes and obtaining more accurate detection results.

[0172] Optionally, a text recognition algorithm can be used to respectively recognize the sub-text content included in at least one target text detection box. Specifically, the embodiment of the present application proposes a text recognition model for text recognition of the target text detection box. To more clearly introduce the text recognition model of the embodiment of the present application, Fig.11 An exemplary text recognition model structure is shown. The text recognition model is composed of a feature extraction module, a recurrent network module, and a sequence recognition module.

[0173] Specifically, for at least one target text detection box, the following operations are respectively performed:

[0174] The feature extraction module extracts features from the image within a target text detection box to obtain detection image features. Based on the detection image features, the recurrent network module determines the temporal features corresponding to the image within a target text detection box. The sequence recognition module recognizes the temporal features to obtain the sub-text content included in a target text detection box.

[0175] Specifically, the feature extraction module includes at least one convolutional module and may also include a downsampling module. The recurrent network module can be an LSTM model or a bidirectional LSTM model.

[0176] For example, as Fig.11 shown, the image within the target text detection box is input into the feature extraction module to obtain detection image features f1. The detection image features f1 are input into the bidirectional LSTM model to obtain temporal features f2. The sequence recognition module recognizes the temporal features f2 to obtain the candidate sub-text content as "-Business - License". Post-process the candidate sub-text content "-Business - License" to obtain the sub-text content "Business License".

[0177] In the embodiment of the present application, the feature extraction module, the recurrent network module and the sequence recognition module in the text recognition model can effectively extract the features corresponding to the image in the target text detection frame and perform recognition. At the same time, the sequence recognition module of the model effectively realizes the recombination of the recognized text and obtains more accurate sub-text content.

[0178] Optionally, the at least one text to be calibrated includes a first text to be calibrated, wherein the first text to be calibrated corresponds to the first reference text. The first text to be calibrated may be a name of a merchant to be calibrated or a name of a legal person to be calibrated.

[0179] Before the above step S204, the following steps are performed for the first text to be calibrated:

[0180] Determine a first text overlap between the first text to be calibrated and the first reference text. Determine a first overlap flag based on a correlation between the first text overlap and an overlap threshold. Determine a first similarity between the first text to be calibrated and the first reference text based on the first text overlap, the first overlap flag, and the text lengths corresponding to the first text to be calibrated and the first reference text.

[0181] Specifically, the number of Chinese characters that are the same between the first text to be calibrated and the first reference text is the first text overlap.

[0182] If the first text overlap is greater than the overlap threshold, the first overlap flag is 1, otherwise it is 0. By comparing the length of the first text to be calibrated with the length of the first reference text, the longer text length and the shorter text length are determined.

[0183] Then, the ratio of the first text overlap to the length of the longer text is calculated to determine the first length ratio. The ratio of the first text overlap to the length of the shorter text is calculated to determine the second length ratio.

[0184] The first length ratio, the second length ratio and the first overlap mark are weighted and added to determine the first similarity between the first text to be calibrated and the first reference text. Each weight can be adjusted according to the specific application scenario and is not limited here.

[0185] Specifically, the first similarity between the first text to be calibrated and the first reference text is expressed by the following formula 1:

[0186]

[0187] Among them, firstSim represents the first similarity between the first text to be calibrated and the first reference text, simLen is the first text overlap, longLen is the long text length, shortLen is the short text length, and simFlag1 is the first overlap flag.

[0188] For example, if the first text overlap is 11, the overlap threshold is set to 2. Since the first text overlap 11 is greater than the overlap threshold 2, the first overlap flag is 1, and the longer text length and the shorter text length are both set to 12. The first similarity calculated by formula (1) is 0.925.

[0189] Optionally, determining the first text overlap between the first text to be calibrated and the first reference text comprises the following steps: Fig.12 As shown:

[0190] Step S1201: determining a feature vector to be calibrated of the first text to be calibrated based on the number of occurrences of each character contained in the first text to be calibrated in the text dictionary.

[0191] Specifically, the text dictionary may be constructed based on the first-level Chinese character library in the GB2312 national standard code, or may be constructed based on the Chinese characters in the first text to be calibrated and the first reference text.

[0192] The length of the feature vector to be calibrated is the same as the length of the text dictionary. The number corresponding to each dimension in the feature vector to be calibrated indicates the number of times the Chinese character corresponding to the dimension appears in the first text to be calibrated.

[0193] Step S1202: Determine a reference feature vector of the first reference text based on the number of occurrences of each character contained in the first reference text in the text dictionary.

[0194] Specifically, the text dictionary corresponding to the first reference text and the text dictionary corresponding to the first text to be calibrated are the same dictionary.

[0195] The length of the reference feature vector is the same as the length of the text dictionary. The number corresponding to each dimension in the reference feature vector indicates the number of times the Chinese character corresponding to the dimension appears in the first reference text, or indicates whether the Chinese character corresponding to the dimension appears in the first reference text.

[0196] For example, the first text to be calibrated is the name of the merchant to be calibrated. The first text to be calibrated is "Gansu Province Wuhu Sihai Trading Co., Ltd.", and the first reference text is "Gansu Province Wuhu Simei Trading Co., Ltd.". The text dictionary constructed based on the first text to be calibrated and the first reference text is {"Gan": 0, "Su": 1, "Province": 2, "Wu": 3, "Hu": 4, "Si": 5, "Hai": 6, "Shang": 7, "Mao": 8, "You": 9, "Limited": 10, "Gong": 11, "Si": 12, "Mei": 13}. The dictionary length is 14, and the lengths of the feature vector to be calibrated and the reference feature vector are both 14.

[0197] The feature vector to be calibrated corresponding to the first text to be calibrated is [1,1,1,1,1,1,1,1,1,1,1,1,0]. The reference feature vector corresponding to the first reference text is [1,1,1,1,1,1,0,1,1,1,1,1,1,1,1].

[0198] Step S1203 : Compare the eigenvalues ​​to be calibrated of each dimension in the eigenvector to be calibrated with the reference eigenvalues ​​of the corresponding dimensions in the reference eigenvector, and obtain the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector.

[0199] For the eigenvalues ​​to be calibrated in each dimension, the following steps are performed respectively: a eigenvalue to be calibrated is compared with a corresponding reference eigenvalue, and based on the comparison result, a minimum eigenvalue is obtained from a eigenvalue to be calibrated and a corresponding reference eigenvalue. Finally, based on the obtained minimum eigenvalues ​​of each dimension, the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector is determined.

[0200] In a possible implementation, the total overlapping dimension between the feature vector to be calibrated and the reference feature vector is determined by adding the minimum eigenvalues ​​of each dimension.

[0201] Another possible implementation manner is to determine the total overlapping dimension between the feature vector to be calibrated and the reference feature vector by weighted addition of the minimum eigenvalues ​​of each dimension.

[0202] For example, the eigenvector to be calibrated is [1,1,1,1,1,1,1,1,1,1,1,1,0], and the reference eigenvector is [1,1,1,1,1,1,0,1,1,1,1,1,1,1,1]. The eigenvalues ​​of each dimension of the eigenvector to be calibrated and the reference eigenvector are compared, and the minimum eigenvalues ​​corresponding to each dimension are [1,1,1,1,1,1,0,1,1,1,1,1,1,0]. The minimum eigenvalues ​​of each dimension are added together to obtain a total overlap of 12.

[0203] Step S1204: determining a first text overlap degree between the first text to be calibrated and the first reference text based on the total overlap dimension.

[0204] Specifically, the total overlapping dimension is the first text overlap between the first text to be calibrated and the first reference text.

[0205] In an embodiment of the present application, the first text overlap degree can effectively record the overlap degree between the first text to be calibrated and the first reference text. When the first text overlap degree is relatively large, the first similarity relatively increases; otherwise, when the first text overlap degree is relatively small, the first similarity relatively decreases. A first overlap flag is determined according to the first text overlap degree, and when calculating the first similarity, the first overlap flag can be used to increase the weight of the first text overlap degree.

[0206] Before the above step S204, at least one text to be calibrated further includes a second text to be calibrated, and the second text to be calibrated corresponds to a second reference text. The second text to be calibrated may be a merchant name to be calibrated or a legal person name to be calibrated.

[0207] For the second text to be calibrated, the following steps are performed, as Fig.13 shown:

[0208] Step S1301, compare the text lengths corresponding to the second text to be calibrated and the second reference text respectively, and based on the comparison result, determine the shortest text length and the second overlap flag.

[0209] Specifically, if the text length of the second text to be calibrated is the same as the text length of the second reference text, determine the second overlap flag as 1, and use the text length of the second text to be calibrated or the second reference text as the shortest text length.

[0210] If the text length of the second text to be calibrated is greater than the text length of the second reference text, determine the second overlap flag as 0, and use the text length of the second reference text as the shortest text length.

[0211] If the text length of the second text to be calibrated is less than the text length of the second reference text, determine the second overlap flag as 0, and use the text length of the second text to be calibrated as the shortest text length.

[0212] Taking the second text to be calibrated as the legal person name to be calibrated as an example, if the second text to be calibrated is "Zhang Sansi" and the second reference text is "Zhang Sisan", then the second overlap flag is 1, and the text lengths of both the second text to be calibrated and the second reference text are 3. Therefore, the shortest text length is 3.

[0213] If the second text to be calibrated is "Zhang Sanfeng" and the second reference text is "Zhang San", then the second overlap flag is 0, and the text length of the second reference text is 2. Therefore, the shortest text length is 2.

[0214] If the second text to be calibrated is "Zhang San" and the second reference text is "Zhang Sanfeng", then the second overlap flag is 0, and the text length of the second text to be calibrated is 2. Therefore, the shortest text length is 2.

[0215] Step S1302: Compare the second text to be calibrated with the second reference text to determine at least one overlapping character between the second text to be calibrated and the second reference text.

[0216] For example, if the second text to be calibrated is "Zhang Sansi" and the second reference text is "Zhang Sisan", then there are three overlapping characters between the second text to be calibrated and the second reference text, namely "Zhang", "Si", and "San".

[0217] Step S1303: Based on the positions of at least one overlapping character in the second text to be calibrated and the second reference text respectively, determine the second text overlap degree.

[0218] Specifically, the second text overlap degree includes a first sub - overlap degree and a second sub - overlap degree.

[0219] For at least one overlapping character, perform the following steps respectively: If the position of an overlapping character in the second text to be calibrated and the second reference text is the same, then regard the overlapping character as a forward overlapping character; otherwise, regard the overlapping character as a reverse overlapping character. Take the number of forward overlapping characters among at least one overlapping character as the first sub - overlap degree. Take the number of reverse overlapping characters among at least one overlapping character as the second sub - overlap degree.

[0220] For example, if the second text to be calibrated is "Zhang Sansi" and the second reference text is "Zhang Sisan", there are three overlapping characters between the second text to be calibrated and the second reference text, namely "Zhang", "San", and "Si". The position of "Zhang" in the second text to be calibrated and the second reference text is the same, so "Zhang" is a forward overlapping character. The position of "San" in the second text to be calibrated and the second reference text is different, so "San" is a reverse overlapping character. The position of "Si" in the second text to be calibrated and the second reference text is different, so "Si" is a reverse overlapping character. There is 1 forward overlapping character and 2 reverse overlapping characters. In summary, the first overlap degree is 1 and the second sub - overlap degree is 2.

[0221] If the second text to be calibrated is "Zhang Sanfeng" and the second reference text is "Zhang San", there are two overlapping characters between the second text to be calibrated and the second reference text, namely "Zhang" and "San". The position of "Zhang" in the second text to be calibrated and the second reference text is the same, so "Zhang" is a forward overlapping character. The position of "San" in the second text to be calibrated and the second reference text is the same, so "San" is a forward overlapping character. In summary, the first overlap degree is 2.

[0222] If the second text to be calibrated is "Zhang San" and the second reference text is "Zhang Sanfeng", there are two overlapping characters in the second text to be calibrated and the second reference text, namely "Zhang" and "San". The positions of "Zhang" in the second text to be calibrated and the second reference text are the same. Therefore, "Zhang" is a positive overlapping character. The positions of "San" in the second text to be calibrated and the second reference text are the same. Therefore, "San" is a positive overlapping character. In summary, the first overlapping degree is 2.

[0223] Step S1304, determine the second similarity between the second text to be calibrated and the second reference text based on the second overlapping flag, the second text overlapping degree, and the shortest text length.

[0224] The second text overlapping degree includes a first sub - overlapping degree and a second sub - overlapping degree. Calculate the ratio of the first sub - overlapping degree to the shortest text length to determine the third length ratio. Calculate the ratio of the second sub - overlapping degree to the shortest text length to determine the fourth length ratio.

[0225] Perform weighted addition on the third length ratio, the fourth length ratio, and the second overlapping flag to determine the second similarity between the second text to be calibrated and the second reference text. Each weight can be adjusted according to the specific application scenario and is not limited here.

[0226] Specifically, the second similarity between the second text to be calibrated and the second reference text is represented by the following formula 2:

[0227]

[0228] Among them, secondSim represents the second similarity between the second text to be calibrated and the second reference text, poseCount is the first sub - overlapping degree, negCount is the second sub - overlapping degree, shortCount is the shortest text length, and simFlag2 is the second overlapping flag.

[0229] For example, the second text to be calibrated is "Zhang San" and the second reference text is "Zhang Er". Therefore, the second overlapping flag is 1, the first sub - overlapping degree is 1, the second sub - overlapping degree is 0, and the shortest text length is 2. By calculating with formula (2), the obtained second similarity is 0.6.

[0230] In the embodiments of the present application, the problem of matching between the second text to be calibrated and the second reference text is fully considered. For example, the problem that the Chinese characters in the second text to be calibrated and the second reference text are the same and in the same position, and the problem that the Chinese characters in the second text to be calibrated and the second reference text are the same but in different positions are considered, which increases the accuracy of calculating the second similarity.

[0231] In order to better explain the embodiment of the present application, a text calibration method provided by the embodiment of the present application is described below in combination with a specific implementation scenario, such as Fig.14 As shown, the following steps are included:

[0232] Step S1401: input the image to be processed into the target detection model to obtain N target text detection boxes.

[0233] Step S1402, set variable i=1.

[0234] Step S1403, determine whether i is less than or equal to N, if so, execute step S1404; otherwise, execute step S1406.

[0235] Step S1404: input the i-th target text detection frame into the text recognition model to obtain the i-th sub-text content.

[0236] Step S1405, set i=i+1, and jump to step S1403.

[0237] Step S1406, determining the unique identifier, the merchant name to be calibrated, and the legal person name to be calibrated from the N sub-text contents.

[0238] Step S1407, based on the unique identifier, obtain the reference merchant name and reference legal person name from the benchmark library.

[0239] Step S1408: determining a first similarity between the merchant name to be calibrated and the reference merchant name.

[0240] Step S1409: determining a second similarity between the legal person name to be calibrated and the reference legal person name.

[0241] Step S1410, determining whether the first similarity is greater than a preset threshold, if so, executing step S1411; otherwise, executing step S1414.

[0242] Step S1411, determine whether the second similarity is greater than a preset threshold, if so, execute step S1412; otherwise, execute step S1414.

[0243] Step S1412: Using the reference merchant name as the calibration result of the merchant name to be calibrated.

[0244] Step S1413: Using the reference legal person name as the calibration result of the legal person name to be calibrated.

[0245] Step S1414, annotate the image to be processed for subsequent verification.

[0246] Specifically, the image to be processed is marked as an erroneous image, and a subsequent manual verification is performed.

[0247] Based on the same technical concept, the embodiment of the present application provides a text calibration device, such as Fig.15 As shown, the device 1500 includes:

[0248] An acquisition module 1501 is used to acquire an image to be processed of a target object;

[0249] An identification module 1502 is used to identify target text content contained in the image to be processed, where the target text content includes a unique identifier of the target object and at least one text to be calibrated;

[0250] A retrieval module 1503 is used to obtain reference texts corresponding to the at least one text to be calibrated from a reference library based on the unique identifier of the target object;

[0251] The comparison module 1504 is configured to use each of the obtained reference texts as the calibration result corresponding to the at least one text to be calibrated if the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold.

[0252] Optionally, the identification module 1502 is specifically configured to:

[0253] Identifying at least one target text detection frame from the image to be processed, and respectively identifying subtext content contained in the at least one target text detection frame;

[0254] The target text content is obtained based on the identified sub-text contents.

[0255] Optionally, the identification module 1502 is further configured to:

[0256] Performing feature extraction on the image to be processed to obtain at least two image features of different sizes;

[0257] fusing the at least two image features of different sizes to obtain the target image feature;

[0258] Based on the target image feature, determining at least one first candidate text detection frame and a confidence level corresponding to each of the at least one first candidate text detection frame;

[0259] Based on the obtained confidence levels, the at least one first candidate text detection frame is screened to obtain at least one target text detection frame.

[0260] Optionally, the identification module 1502 is further configured to:

[0261] Iteratively fusing the at least two image features of different sizes until the at least two image features of different sizes are fused into one fused feature, obtaining a target fused feature, and performing feature extraction on the target fused feature to obtain the target image feature;

[0262] Each iterative fusion includes the following steps:

[0263] Selecting the image feature with the smallest size from among the image features retained after the previous iterative fusion as the reference feature;

[0264] Based on the size of the reference feature, the fusion feature obtained by the previous iterative fusion is adjusted, and the fusion feature obtained after the size adjustment is fused with the reference feature to obtain the fusion feature of this iterative fusion.

[0265] Optionally, the identification module 1502 is further configured to:

[0266] Acquire, from the at least one first candidate text detection frame, a second candidate text detection frame whose confidence is greater than or equal to a confidence threshold;

[0267] Based on the obtained overlapping properties between the second candidate text detection frames, the at least one target text detection frame is obtained from the second candidate text detection frames.

[0268] Optionally, the identification module 1502 is further configured to:

[0269] Sorting the second candidate text detection boxes in descending order of confidence to obtain a sorting result;

[0270] According to the sorting result, a reference detection frame is selected from each of the retained second candidate text detection frames in turn, and based on the overlapping properties between the selected reference detection frame and the other retained second candidate text detection frames, the other retained second candidate text detection frames are screened to obtain the at least one target text detection frame.

[0271] Optionally, the identification module 1502 is further configured to:

[0272] For the second candidate text detection box ranked at the Nth position in the sorting result, the following screening steps are performed:

[0273] Determine the intersection-and-union ratios between the Nth second candidate text detection frame and other retained second candidate text detection frames respectively; wherein N is an integer greater than 0;

[0274] The second candidate text detection frames whose intersection-over-union ratio is greater than a preset threshold value are deleted from the other retained second candidate text detection frames.

[0275] Optionally, the identification module 1502 is further configured to:

[0276] For the at least one target text detection frame, perform the following operations respectively:

[0277] Extract features of an image within a target text detection frame to obtain detection image features;

[0278] Based on the detected image features, determining the temporal features corresponding to the image within the target text detection frame;

[0279] The temporal features are identified to obtain the sub-text content contained in the target text detection frame.

[0280] Optionally, the at least one text to be calibrated includes a first text to be calibrated, and the first text to be calibrated corresponds to a first reference text;

[0281] The similarity calculation module 1505 is also included, and the similarity calculation module 1505 is specifically used for:

[0282] If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as the calibration result corresponding to the at least one text to be calibrated,

[0283] determining a first text overlap between the first to-be-calibrated text and the first reference text;

[0284] Determining a first overlapping mark based on the correlation between the first text overlapping degree and the overlapping degree threshold;

[0285] A first similarity between the first text to be calibrated and the first reference text is determined according to the first text overlap degree, the first overlap mark, and the text lengths corresponding to the first text to be calibrated and the first reference text.

[0286] Optionally, the similarity calculation module 1505 is further used for:

[0287] Determining a feature vector to be calibrated of the first text to be calibrated based on the number of occurrences of each character contained in the first text to be calibrated in the text dictionary;

[0288] Determining a reference feature vector of the first reference text based on the number of occurrences of each word contained in the first reference text in a text dictionary;

[0289] Compare the eigenvalues ​​to be calibrated of each dimension in the eigenvector to be calibrated with the reference eigenvalues ​​of the corresponding dimensions in the reference eigenvector, respectively, to obtain the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector;

[0290] Based on the total overlap dimension, a first text overlap degree between the first to-be-calibrated text and the first reference text is determined.

[0291] Optionally, the similarity calculation module 1505 is further used for:

[0292] For the eigenvalues ​​to be calibrated in each dimension, the following steps are performed respectively: comparing one eigenvalue to be calibrated with a corresponding reference eigenvalue, and based on the comparison result, obtaining a minimum eigenvalue from the one eigenvalue to be calibrated and the corresponding reference eigenvalue;

[0293] Based on the obtained minimum eigenvalues ​​of the various dimensions, the total overlapping dimension between the feature vector to be calibrated and the reference feature vector is determined.

[0294] Optionally, the at least one text to be calibrated further includes a second text to be calibrated, and the second text to be calibrated corresponds to the second reference text;

[0295] The similarity calculation module 1505 is also used for:

[0296] Comparing the text lengths corresponding to the second text to be calibrated and the second reference text, and determining the shortest text length and the second overlap mark based on the comparison result;

[0297] comparing the second text to be calibrated with the second reference text to determine at least one overlapping word between the second text to be calibrated and the second reference text;

[0298] Determine a second text overlap degree based on the positions of the at least one overlapping character in the second text to be calibrated and the second reference text respectively;

[0299] Based on the second overlap mark, the second text overlap degree, and the shortest text length, a second similarity between the second text to be calibrated and the second reference text is determined.

[0300] Optionally, the similarity calculation module 1505 is further used for:

[0301] If the text length of the second text to be calibrated is the same as the text length of the second reference text, the second overlap flag is determined to be 1, and the text length of the second text to be calibrated or the second reference text is used as the shortest text length;

[0302] If the text length of the second text to be calibrated is greater than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second reference text is used as the shortest text length;

[0303] If the text length of the second text to be calibrated is less than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second text to be calibrated is used as the shortest text length.

[0304] Optionally, the second text overlap includes a first sub-overlap and a second sub-overlap;

[0305] The similarity calculation module 1505 is also used for:

[0306] For the at least one overlapping character, the following steps are respectively performed: if the position of an overlapping character in the second to-be-calibrated text and the second reference text is the same, the overlapping character is used as a forward overlapping character, otherwise the overlapping character is used as a reverse overlapping character;

[0307] The number of positive overlapping characters in the at least one overlapping character is used as a first sub-overlapping degree;

[0308] The number of reverse overlapping characters in the at least one overlapping character is used as the second sub-overlapping degree.

[0309] Optionally, the at least one text to be calibrated includes: a merchant name to be calibrated and a legal person name to be calibrated, and the reference text corresponding to each of the at least one text to be calibrated includes: a reference merchant name and a reference legal person name.

[0310] Based on the same technical concept, the embodiment of the present application provides a computer device, which may be a terminal or a server, such as Fig.16 As shown, it includes at least one processor 1601 and a memory 1602 connected to the at least one processor. The specific connection medium between the processor 1601 and the memory 1602 is not limited in the embodiment of the present application. Fig.16 For example, the processor 1601 and the memory 1602 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0311] In the embodiment of the present application, the memory 1602 stores instructions that can be executed by at least one processor 1601 , and the at least one processor 1601 can execute the steps included in the above-mentioned text calibration method by executing the instructions stored in the memory 1602 .

[0312] Among them, the processor 1601 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, and perform text calibration by running or executing instructions stored in the memory 1602 and calling data stored in the memory 1602. Optionally, the processor 1601 may include one or more processing units, and the processor 1601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1601. In some embodiments, the processor 1601 and the memory 1602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0313] Processor 1601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0314] Memory 1602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 1602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0315] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned text calibration method.

[0316] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0317] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0318] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0319] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0320] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A text calibration method, characterized in that: include: Obtain an image to be processed of a target object; Identifying target text content contained in the image to be processed, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated; Based on the unique identifier of the target object, obtaining reference texts corresponding to the at least one text to be calibrated from a reference library; If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, each reference text obtained is used as a calibration result corresponding to the at least one text to be calibrated; The at least one text to be calibrated includes a first text to be calibrated, and the first text to be calibrated corresponds to a first reference text; If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, before using each reference text obtained as the calibration result corresponding to the at least one text to be calibrated, the method further includes: determining a first text overlap between the first to-be-calibrated text and the first reference text; Determining a first overlapping mark based on the correlation between the first text overlapping degree and the overlapping degree threshold; Determining a first similarity between the first text to be calibrated and the first reference text according to the first text overlap degree, the first overlap mark, and the text lengths corresponding to the first text to be calibrated and the first reference text; The determining a first text overlap between the first to-be-calibrated text and the first reference text includes: Determining a feature vector to be calibrated of the first text to be calibrated based on the number of occurrences of each character contained in the first text to be calibrated in the text dictionary; Determining a reference feature vector of the first reference text based on the number of occurrences of each word contained in the first reference text in a text dictionary; Compare the eigenvalues ​​to be calibrated of each dimension in the eigenvector to be calibrated with the reference eigenvalues ​​of the corresponding dimensions in the reference eigenvector, respectively, to obtain the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector; Based on the total overlap dimension, a first text overlap degree between the first to-be-calibrated text and the first reference text is determined.

2. The method according to claim 1, characterized in that The identifying target text content contained in the image to be processed includes: Identifying at least one target text detection frame from the image to be processed, and respectively identifying subtext content contained in the at least one target text detection frame; The target text content is obtained based on the identified sub-text contents.

3. The method according to claim 2, characterized in that The step of identifying at least one target text detection frame from the image to be processed includes: Performing feature extraction on the image to be processed to obtain at least two image features of different sizes; fusing the at least two image features of different sizes to obtain target image features; Based on the target image feature, determining at least one first candidate text detection frame and a confidence level corresponding to each of the at least one first candidate text detection frame; Based on the obtained confidence levels, the at least one first candidate text detection frame is screened to obtain at least one target text detection frame; and the at least two image features of different sizes are fused to obtain the target image features, including: Iteratively fusing the at least two image features of different sizes until the at least two image features of different sizes are fused into one fused feature, obtaining a target fused feature, and performing feature extraction on the target fused feature to obtain the target image feature; Each iterative fusion includes the following steps: Among the image features retained after the previous iterative fusion, the image feature with the smallest size is selected as the reference feature; Based on the size of the reference feature, adjusting the fusion feature obtained by the previous iterative fusion, and fusing the fusion feature obtained after the size adjustment with the reference feature to obtain the fusion feature of this iterative fusion; based on each confidence obtained, screening the at least one first candidate text detection frame to obtain at least one target text detection frame, including: Acquire, from the at least one first candidate text detection frame, a second candidate text detection frame whose confidence is greater than or equal to a confidence threshold; Based on the obtained overlapping attributes between the respective second candidate text detection frames, the at least one target text detection frame is obtained from the respective second candidate text detection frames; the at least one target text detection frame is obtained from the respective second candidate text detection frames based on the obtained overlapping attributes between the respective second candidate text detection frames, including: Sorting the second candidate text detection boxes in descending order of confidence to obtain a sorting result; According to the sorting result, a reference detection frame is selected from each of the retained second candidate text detection frames in turn, and based on the overlapping attribute between the selected reference detection frame and the other retained second candidate text detection frames, the other retained second candidate text detection frames are screened to obtain the at least one target text detection frame; for the second candidate text detection frame ranked at the Nth position in the sorting result, the following screening steps are performed: Determine the intersection-and-union ratios between the Nth second candidate text detection frame and other retained second candidate text detection frames respectively; wherein N is an integer greater than 0; The second candidate text detection frames whose intersection-over-union ratio is greater than a preset threshold value are deleted from the other retained second candidate text detection frames.

4. The method according to claim 2, characterized in that Respectively identifying subtext content contained in the at least one target text detection frame, including: For the at least one target text detection frame, perform the following operations respectively: Extract features of an image within a target text detection frame to obtain detection image features; Based on the detected image features, determining the temporal features corresponding to the image within the target text detection frame; The temporal features are identified to obtain the sub-text content contained in the target text detection frame.

5. The method according to claim 1, characterized in that The step of comparing the eigenvalues ​​to be calibrated of each dimension in the eigenvector to be calibrated with the reference eigenvalues ​​of the corresponding dimensions in the reference eigenvector to obtain the total overlapping dimension between the eigenvector to be calibrated and the reference eigenvector, comprises: For the eigenvalues ​​to be calibrated in each dimension, the following steps are performed respectively: comparing one eigenvalue to be calibrated with a corresponding reference eigenvalue, and based on the comparison result, obtaining a minimum eigenvalue from the one eigenvalue to be calibrated and the corresponding reference eigenvalue; Based on the obtained minimum eigenvalues ​​of the various dimensions, the total overlapping dimension between the feature vector to be calibrated and the reference feature vector is determined.

6. The method according to claim 1, characterized in that The at least one text to be calibrated further includes a second text to be calibrated, and the second text to be calibrated corresponds to the second reference text; If the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold, before using each reference text obtained as the calibration result corresponding to the at least one text to be calibrated, the method further includes: Comparing the text lengths corresponding to the second text to be calibrated and the second reference text, and determining the shortest text length and the second overlap mark based on the comparison result; comparing the second text to be calibrated with the second reference text to determine at least one overlapping word between the second text to be calibrated and the second reference text; Determine a second text overlap degree based on the positions of the at least one overlapping character in the second text to be calibrated and the second reference text respectively; Based on the second overlap mark, the second text overlap degree, and the shortest text length, a second similarity between the second text to be calibrated and the second reference text is determined.

7. The method according to claim 6, characterized in that Determining the shortest text length and the second overlap mark based on the comparison result includes: If the text length of the second text to be calibrated is the same as the text length of the second reference text, the second overlap flag is determined to be 1, and the text length of the second text to be calibrated or the second reference text is used as the shortest text length; If the text length of the second text to be calibrated is greater than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second reference text is used as the shortest text length; If the text length of the second text to be calibrated is less than the text length of the second reference text, the second overlap flag is determined to be 0, and the text length of the second text to be calibrated is used as the shortest text length.

8. The method according to claim 6, characterized in that The second text overlap includes a first sub-overlap and a second sub-overlap; The determining the second text overlap degree based on the positions of the at least one overlapping character in the second text to be calibrated and the second reference text, respectively, comprises: For the at least one overlapping character, the following steps are respectively performed: if the position of an overlapping character in the second to-be-calibrated text and the second reference text is the same, the overlapping character is used as a forward overlapping character, otherwise the overlapping character is used as a reverse overlapping character; The number of positive overlapping characters in the at least one overlapping character is used as a first sub-overlapping degree; The number of reverse overlapping characters in the at least one overlapping character is used as the second sub-overlapping degree.

9. The method according to any one of claims 1 to 8, characterized in that: The at least one text to be calibrated includes: a merchant name to be calibrated and a legal person name to be calibrated, and the reference text corresponding to each of the at least one text to be calibrated includes: a reference merchant name and a reference legal person name.

10. A text calibration device, characterized in that: include: An acquisition module, used for acquiring an image to be processed of a target object; A recognition module, used for recognizing target text content contained in the image to be processed, wherein the target text content includes a unique identifier of the target object and at least one text to be calibrated; A retrieval module, configured to obtain, from a reference library, reference texts corresponding to the at least one text to be calibrated based on the unique identifier of the target object; A comparison module, configured to use each reference text obtained as a calibration result corresponding to the at least one text to be calibrated if the similarity between the at least one text to be calibrated and the corresponding reference text is greater than a preset threshold; The at least one text to be calibrated includes a first text to be calibrated, and the first text to be calibrated corresponds to a first reference text; and further includes a similarity calculation module, wherein the similarity calculation module is specifically used to: if the similarities between the at least one text to be calibrated and the corresponding reference text are greater than a preset threshold, determine the first text overlap between the first text to be calibrated and the first reference text before using each reference text obtained as the calibration result corresponding to the at least one text to be calibrated; determine a first overlap mark based on the correlation between the first text overlap and the overlap threshold; determine the first similarity between the first text to be calibrated and the first reference text according to the first text overlap, the first overlap mark, and the text lengths corresponding to the first text to be calibrated and the first reference text respectively; The similarity calculation module is also used to: determine the feature vector to be calibrated of the first text to be calibrated based on the number of occurrences of each character contained in the first text to be calibrated in the text dictionary; determine the reference feature vector of the first reference text based on the number of occurrences of each character contained in the first reference text in the text dictionary; compare the feature values ​​to be calibrated of each dimension in the feature vector to be calibrated with the reference feature values ​​of the corresponding dimensions in the reference feature vector, respectively, to obtain the total overlapping dimension between the feature vector to be calibrated and the reference feature vector; and determine the first text overlap between the first text to be calibrated and the first reference text based on the total overlapping dimension.

11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 9.

Citation Information

Patent Citations

  • A paper relevance quantification method based on reference list overlap degree

    CN109376238A

  • Image text recognition method and device, computer equipment and computer storage medium

    CN111275038A

  • Order information correction method and device, equipment and storage medium

    CN112395874A

  • Skew detecting method and character recognizing device

    JP1992335486A

  • Device, method and program for extracting document area from image

    JP2010231686A