Optical character sequence recognition method and device

Abstract

The invention provides a character sorting method based on a binary tree, a character sorting device based on the binary tree, an optical character sequence positioning and recognition method, an optical character sequence positioning and recognition device, electronic equipment and a computer readable storage medium. The character sorting method based on the binary tree comprises the following steps: performing pattern collection in a region comprising a target character sequence to obtain a pattern set; traversing each pattern in the pattern set, and placing the patterns in the same row or column in the same binary tree; and sequentially identifying patterns at each node from the binary tree to obtain a target character sequence.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition, and more specifically to a binary tree-based character sorting method, a binary tree-based character sorting device, an optical character sequence recognition method, an optical character sequence recognition device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Optical character recognition is the process of analyzing image files with text data to obtain text and layout information. In terms of application scenarios, recognizing text information in natural scene images is much more difficult than recognizing text in scanned document images. This is because: characters have different sizes, fonts, colors, brightness, contrast, etc.; text lines can be horizontal, vertical, curved, rotated, twisted, etc.; the text area in the image may be deformed (perspective, affine transformation), incomplete, blurred, etc.; the background of natural scene images is extremely diverse, such as text on a flat, curved or wrinkled surface, complex interfering textures near the text area, or textures similar to text in the non-text area.

[0003] At present, image text detection and recognition technology has a wide range of application scenarios, such as business card recognition, menu recognition, express delivery note recognition, ID card recognition, business license recognition, bank card recognition, license plate recognition, road sign recognition, product packaging bag recognition, conference whiteboard recognition, advertising main words recognition, test paper recognition, document recognition, carriage container number recognition, etc.

[0004] However, current text detection and recognition technologies have difficulty in identifying character sequences in small-sized areas in large-sized natural scenes or complex scene images.

[0005] Therefore, a technology is needed that can recognize character sequences in small-sized areas in large-sized natural scenes or complex scene images. Summary of the invention

[0006]

Technical solution

[0007] The present application proposes multiple aspects to solve the above problems.

[0008] In the first aspect of the present application, a binary tree-based character sorting method includes: collecting patterns in an area including a target character sequence to obtain a pattern set; traversing each pattern in the pattern set and placing patterns in the same row or column in the same binary tree; and sequentially identifying patterns at each node in the binary tree to obtain the target character sequence.

[0009] According to the first aspect, placing patterns in the same row or column in the same binary tree may include: the pattern placed at each node in the left subtree of the binary tree is a pattern before a baseline pattern placed at the root node of the binary tree, and wherein the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the baseline pattern in the row or column where it is located, and wherein the pattern placed at each node in the right subtree of the binary tree is a pattern after the baseline pattern placed at the root node of the binary tree, and wherein the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the baseline pattern in the row or column where it is located.

[0010] According to the first aspect, traversing each pattern in the pattern set includes: determining the probability that each pattern in the pattern set matches a character pre-stored in a database; determining a pattern whose probability is greater than a first predetermined threshold as a character; forming a character set using the determined characters; and taking the character with the highest probability in the character set as the starting character, traversing each character in the character set in order of probability from large to small.

[0011] According to the first aspect of the present application, the patterns placed at the nodes in the binary tree are all characters, and the reference pattern placed at the root node in the binary tree is the starting character.

[0012] According to the first aspect of the present application, placing characters in the same row or column in the same binary tree includes: calculating the angle between the traversed character and the center of mass line of the starting character and a predetermined axis; in response to the calculated angle being less than or equal to a predetermined angle threshold, determining that the traversed character and the starting character are in the same row or column, and placing the traversed character in a binary tree with the starting character as the root node; and in response to the calculated angle being greater than the predetermined angle threshold, determining that the traversed character and the starting character are in different rows or columns, and placing the traversed character in a new binary tree.

[0013] According to the first aspect of the present application, after determining that the traversed character is in the same row or column as the starting character, the following operations are also performed: calculating a first distance between a first point in the minimum rectangular bounding box surrounding the starting character and a second point in the minimum rectangular bounding box surrounding the area; calculating a second distance between a third point in the minimum rectangular bounding box surrounding the traversed character and the second point, wherein the position of the third point in the minimum rectangular bounding box surrounding the traversed character corresponds to the position of the first point in the minimum rectangular bounding box surrounding the starting character; and determining whether the traversed character is before or after the starting character based on the first distance and the second distance.

[0014] According to a first aspect of the present application, in response to the calculated angle being greater than the predetermined angle threshold, determining that the traversed character and the starting character are in different rows or columns includes: when there are multiple rows or columns in the area, there are multiple corresponding predetermined angle thresholds θ i , where i = {1, 2, ..., N}, and where, when the calculated angle is greater than θ i and less than or equal to θ i+1 , it is determined that the traversed character is in the i+1th row starting from the row or column where the starting character is located, and the traversed character is correspondingly placed in the i-th new binary tree.

[0015] According to the first aspect of the present application, the arrangement of characters in the new binary tree is the same as the arrangement of characters in the binary tree with the starting character as the root node.

[0016] According to the first aspect of the present application, the first point is one of the following items: center point, upper left corner point, centroid point; and the second point is one of the following items: upper left corner point, upper right corner point, lower left corner point, lower right corner point.

[0017] In a second aspect of the present application, a binary tree-based character sorting device comprises:

[0018] A collection unit is configured to collect patterns in an area including a target character sequence to obtain a pattern set; and a first processing unit is configured to: traverse each pattern in the pattern set and place the patterns in the same row or column in the same binary tree, wherein the pattern placed at each node in the left subtree of the binary tree is a pattern before a reference pattern placed at the root node of the binary tree, and wherein the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located, and wherein the pattern placed at each node in the right subtree of the binary tree is a pattern after the reference pattern placed at the root node of the binary tree, and wherein the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located; and sequentially identify the patterns at each node in the binary tree to obtain the target character sequence.

[0019] In a third aspect of the present application, an optical character sequence positioning and recognition method comprises: positioning an area including a target character sequence in an input image; and identifying the target character sequence in the area using the binary tree-based character sorting method according to the first aspect.

[0020] According to the third aspect of the present application, the method also includes: when performing a positioning operation, performing at least one level of area positioning based on the characteristics of the target character sequence, the size comparison between the input image and the area including the target character sequence, and the complexity of the scene.

[0021] In a fourth aspect of the present application, an optical character sequence positioning and recognition device comprises: a positioning unit configured to locate an area including a target character sequence in an input image; and a second processing unit, utilizing a binary tree-based character sorting method according to any one of claims 1 to 8 to identify the target character sequence in the area.

[0022] In a fifth aspect of the present application, an electronic device comprises: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to the first aspect or according to the third aspect.

[0023] In a sixth aspect of the present application, a computer-readable storage medium stores executable instructions thereon, which, when executed by a processor, enables the processor to implement the method according to the first aspect or according to the third aspect.

[0024]

Technical Effect

[0025] Compared with the previous character sequence recognition methods, the present invention proposes a method for obtaining character sequences in small-sized areas in large-sized natural scenes or complex scene images. This method can design corresponding regional positioning methods according to the characteristics of the actual scene itself, and is not limited to obtaining positioning methods such as specifying a fixed position or angle of the image acquisition hardware. Flexible application will simplify the hardware requirements in the design of the entire recognition system. In addition, this method is not limited to one-time positioning, and multi-level positioning is performed according to the characteristics of the character area, which simplifies the technical difficulty of recognition processing for the entire image, simplifies the algorithm model, and improves computing resource utilization and timeliness.

[0026] In addition, the technical solution of the present invention can be applied to a variety of character sequence arrangements, such as horizontal, vertical, inclined, single row, multiple rows, single column, multiple columns, etc.; character sequence recognition in an area of ​​corresponding difficulty can be designed according to the complexity of the arrangement.

[0027] Taking container number recognition as an example, the present invention has a single-channel recognition effect of over 98%, regional positioning of about 0.2 seconds, and single-image recognition of less than 0.5 seconds, which can meet the container number recognition tasks in various application scenarios (customs, checkpoints, etc.); taking customs declaration number recognition as an example, the recognition effect of customs declaration number in scanned image files reaches over 96%, and single-image recognition is less than 0.5 seconds. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other embodiments and features of the present disclosure will become more apparent by describing in detail embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0029] Figure 1 The system architecture of the character recognition method according to the embodiment of the present disclosure is schematically shown;

[0030] Figure 2 A flowchart of a binary tree-based character sorting method according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 3 The area 300 including the target character sequence and the corresponding binary trees are schematically shown.

[0032] Figure 4 A flowchart for determining a root node and determining an order of patterns to be traversed according to an embodiment of the present disclosure is schematically shown;

[0033] Figure 5 Schematically shows a binary tree after non-character patterns are removed according to an embodiment of the present disclosure;

[0034] Figure 6 A flowchart of placing characters in the same row or column in the same binary tree according to an embodiment of the present disclosure is schematically shown;

[0035] Figure 7 A flowchart schematically shows, according to an embodiment of the present disclosure, further determining the position of the traversed character in the binary tree after determining that the traversed character is in the same row or column as the starting character;

[0036] Figure 8 A flowchart schematically shows further processing when it is determined that the traversed character and the starting character are in different rows or columns according to an embodiment of the present disclosure;

[0037] Fig. 9 A block diagram of a binary tree-based character sorting device according to an embodiment of the present disclosure is schematically shown;

[0038] Fig.10 A flowchart of an optical character sequence positioning and recognition method according to an embodiment of the present disclosure is schematically shown;

[0039] Fig.11 Schematically illustrates a relatively simple scenario of character sequence region positioning according to an embodiment of the present disclosure;

[0040] Fig.12 Schematically illustrates an embodiment of the present disclosure. Fig.11 The sequence including the target characters located in the larger size image shown in and the target character sequence finally recognized;

[0041] Fig.13 Schematically illustrates a relatively complex scene of character sequence region positioning according to an embodiment of the present disclosure;

[0042] Fig.14 Schematically illustrates an embodiment of the present disclosure. Fig.13 The box area where the customs declaration number is located in the image shown in FIG;

[0043] Fig.15 Schematically illustrates an embodiment of the present disclosure. Fig.14 The customs declaration number box located at the second level in the box area shown in and the final target character sequence identified from the box;

[0044] Fig.16 The program for implementing the above optical character sequence positioning and recognition method is schematically shown;

[0045] Fig.17 A block diagram schematically shows a device for implementing the above optical character sequence positioning and recognition method; and

[0046] Fig.18 A block diagram of an electronic device suitable for implementing a binary tree-based character sorting method and an optical character sequence positioning and recognition method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0047] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not have to be used to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, materials or methods are not specifically described.

[0048] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Thus, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. Furthermore, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any appropriate combinations and / or subcombinations.

[0049] It should be understood that when an element is said to be "coupled to" or "connected to" another element, it can be directly coupled or connected to the other element or there may be intermediate elements. On the contrary, when an element is said to be "directly coupled to" or "directly connected to" another element, there are no intermediate elements.

[0050] Furthermore, as used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0051] It will be understood that a noun corresponding to a term in the singular form may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B or C" may include all possible combinations of items listed together with a corresponding one of the plurality of phrases. As used herein, terms such as "first" and "second" or "first" and "second" may be used to simply distinguish a corresponding component from another component and do not limit the components in other respects (e.g., importance or order).

[0052] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0053] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or replacement forms for the corresponding embodiments. Unless otherwise clearly defined herein, all terms will be given their broadest possible interpretation, including the meanings implied in the specification and the meanings understood by those skilled in the art and / or defined in dictionaries, papers, etc.

[0054] In addition, it should be understood by those skilled in the art that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale. For the description of the drawings, similar reference numerals may be used to refer to similar or related elements. The present disclosure will be described exemplarily with reference to the drawings below.

[0055] It should be noted that conventional character recognition methods include the following aspects.

[0056] First, the Canny algorithm is used to perform rough positioning in the entire image, and then the MSER algorithm is used for fine positioning. The binary features and color space are combined to extract the area, the projection method is used for character segmentation, and the characters are normalized and classified using the CNN network.

[0057] The disadvantages of this method are that both Canny and MSER algorithms are affected by lighting and background, are limited by color space, and are restricted to the use of fixed colors.

[0058] Perform color detection on each pixel in the entire image to obtain the pixel corresponding to the color of the road sign in the image; obtain at least one first connected area of ​​the same color, and filter to obtain at least one second connected area (that is, filter the first connected area according to the characteristics of the road sign image to obtain at least one second connected area); then use the SVM algorithm to filter out the road sign area from the second connected area, and classify and recognize the characters in the filtered area.

[0059] The disadvantage of this method is that it is limited by the color space and is restricted to the use of limited fixed colors.

[0060] The MSER algorithm is used to segment the entire image to obtain candidate character regions, and the obtained characters are merged through rule filtering; the characters are connected into candidate text lines through hierarchical clustering, and then filtered to obtain the final text lines; the single characters in the text lines are recognized using a deep learning CNN network to obtain character recognition results.

[0061] The disadvantage of this method is that the MSER algorithm is easily affected by lighting. In particular, when there is uneven lighting, occlusion, missing characters, blur, etc., using the MSER algorithm to locate the character area often results in missing characters.

[0062] The recognition system consists of two parts: the detection module and the recognition module. In the detection module, two detection methods, the MSER algorithm and the deep neural network CTPN, are used and combined. The recognition module also uses two methods, one is to use ResNet to recognize the segmented single characters, and the other is to use CRNN to recognize the entire box number text line and then directly output the string.

[0063] The disadvantages of this method are: CTPN is suitable for situations where characters are arranged horizontally, and there is currently no solution to use CTPN in situations where the characters are greatly tilted or arranged vertically; the CRNN method in the recognition module also cannot recognize vertically distributed character sequences; the system integrates multiple algorithms, so the timeliness of the recognition system cannot be guaranteed.

[0064] Acquire real-time camera images in large scenes; extract image features; use the ResNet residual neural network model to detect image features; if the detection result belongs to the preset classification result, extract the character area coordinate information in the image feature; use the LSTM network model to recognize the intercepted area coordinate image to obtain text information.

[0065] The disadvantages of this method are: the model is complex, it takes up a lot of resources, and timeliness cannot be guaranteed; the LSTM algorithm mainly processes single-line or single-column text data. According to statistics, the actual box number distribution is horizontal and vertical. The horizontal distribution is divided into one row, two rows, and multiple rows; the vertical distribution is divided into one column and two columns. This method cannot meet all applications.

[0066] Use the character recognition system to identify the digital amount in the bill, and compare different types of characters with predetermined standards (according to the characteristics of different fields in the bill, the text field and the Arabic field correspond one to one. For example, one thousand six hundred and fifty-three, 1653, 1 corresponds to one thousand, 6 corresponds to six hundred, 5 corresponds to fifty, and 3 corresponds to three).

[0067] The disadvantage of this method is that it is necessary to place the designated effective area of ​​the bill on the recognition device, and there is no intelligent positioning of the required text in the complex text.

[0068] The conventional character recognition technology mentioned above recognizes characters in the area when the character area has been located or the background is relatively simple, and the characters are mostly arranged horizontally. For vertical, inclined or more complex arrangements, the conventional character recognition technology mentioned above cannot correctly and effectively detect and recognize characters.

[0069] In order to solve the problems mentioned above, an embodiment of the present disclosure provides a character sorting method based on a binary tree, which may include the following operations: collecting patterns in an area including a target character sequence to obtain a pattern set; traversing each pattern in the pattern set and placing patterns in the same row or column in the same binary tree; and sequentially identifying the patterns at each node in the binary tree to obtain the target character sequence.

[0070] Another embodiment of the present disclosure provides a binary tree-based character sorting device, which may include: a collection unit, configured to collect patterns in an area including a target character sequence to obtain a pattern set; and a first processing unit, configured to: traverse each pattern in the pattern set, place patterns in the same row or column in the same binary tree; and sequentially identify the patterns at each node in the binary tree to obtain the target character sequence.

[0071] It should be noted that, according to the characteristics of the binary tree, when sequentially identifying the patterns at each node in the binary tree, start from the root node of the left subtree, read upward to the root node sequentially, and then read each node in the right subtree sequentially from the root node downward.

[0072] It should also be noted that when placing patterns in the same row or column in the same binary tree, the pattern placed at each node in the left subtree of the binary tree is a pattern that precedes the baseline pattern placed at the root node of the binary tree, and wherein the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the baseline pattern in the row or column where it is located, and wherein the pattern placed at each node in the right subtree of the binary tree is a pattern that follows the baseline pattern placed at the root node of the binary tree, and wherein the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the baseline pattern in the row or column where it is located.

[0073] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with specific embodiments.

[0074] Character sorting method based on binary tree and device for implementing the method

[0075] Figure 1 The system architecture 100 of the character recognition method according to the embodiment of the present disclosure is schematically shown.

[0076] like Figure 1 As shown, when recognizing a target character sequence in a large-scale natural scene or complex scene image, the area A including the target character sequence occupies a very small part of the input image.

[0077] from Figure 1 It can be seen from the enlarged view of area A in the input image 101 on the right side that the characters in the area including the target characters may be deformed, have special formats (for example, underlined), be distributed in multiple rows or columns, etc. In this case, the use of conventional character recognition methods will result in unsatisfactory character recognition results.

[0078] In order to solve this problem, large-scale natural scene or complex scene images ( Figure 1 The input image 101 in the image is input to the central control unit 102, and the central control unit 102 applies the binary tree-based character sorting method provided by the present application described in detail below to process the input image and recognize the characters in the area, and finally outputs the target character sequence from the central control unit 102.

[0079] It should be noted that the region including the target character sequence in the input image may include various other patterns (such as Fig.11 and Fig.13 ).

[0080] Figure 1 The central control unit 102 in the embodiment may be any suitable electronic device such as a desktop computer, a tablet computer, a laptop computer, a smart phone, etc. suitable for executing the method provided in the present application.

[0081] Figure 1 The central control unit 102 in the system can be configured to perform the following operations: collect patterns in an area including a target character sequence to obtain a pattern set; traverse each pattern in the pattern set and place patterns in the same row or column in the same binary tree; and sequentially identify the patterns at each node in the binary tree to obtain the target character sequence.

[0082] exist Figure 1When the central control unit 102 places patterns in the same row or column in the same binary tree, the pattern placed at each node in the left subtree of the binary tree is a pattern before the reference pattern placed at the root node of the binary tree, and wherein the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located, and wherein the pattern placed at each node in the right subtree of the binary tree is a pattern after the reference pattern placed at the root node of the binary tree, and wherein the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located.

[0083] Figure 1 The central control unit 102 may include a processor (not shown), a memory (not shown), a communication module (not shown), and the like.

[0084] The processor of the central control unit 102 may be configured to implement the method provided in the present application.

[0085] The memory of the central control unit 102 may store various data and instructions related to various operations performed by the central control unit 102. For example, various pre-stored characters, various positioning models, various recognition models, various predetermined thresholds, instructions, programs, etc. may be stored.

[0086] The communication module in the central control unit 102 may support establishment of a direct (eg, wired) communication channel or a wireless communication channel between the central control unit 102 and an external electronic device, and perform communication via the established communication channel.

[0087] The communication module may include one or more communication processors capable of operating independently from a processor (e.g., an application processor (AP)) and supporting wired communication and / or wireless communication. According to an embodiment of the present disclosure, the communication module may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate with an external electronic device via a first network (e.g., a short-range communication network such as Bluetooth, wireless fidelity (Wi-Fi) direct connection, or infrared data association (IrDA)) or a second network (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or these various types of communication modules may be implemented as multiple components (e.g., multiple chips) separated from each other. The wireless communication module may identify and authenticate the electronic device in a communication network (such as the first network or the second network) using subscriber information (eg, International Mobile Subscriber Identity (IMSI)) stored in the subscriber identification module.

[0088] In addition, the central control unit 102 may also include a display, a microphone, etc., for displaying or broadcasting information such as the recognized target character sequence.

[0089] Figure 2 The flowchart of the binary tree-based character sorting method according to an embodiment of the present disclosure is schematically shown.

[0090] like Figure 2 As shown, the method includes the following operations.

[0091] In operation S201, the central control unit 102 generates a large-scale natural scene or complex scene image ( Figure 1 Pattern collection is performed in an area including a target character sequence in an input image 101) to obtain a pattern set.

[0092] In operation S203 , the central control unit 102 traverses each pattern in the pattern set and places the patterns in the same row or column in the same binary tree.

[0093] Exemplarily, the area including the target character sequence may include patterns of multiple rows and / or multiple columns. At this time, when identifying in units of rows, the patterns in the same rows can be placed in the same binary tree. When identifying in units of columns, the patterns in the same columns can be placed in the same binary tree. In this case, the central control unit 102 can construct multiple binary trees corresponding to each row or column in the multiple rows or columns.

[0094] Those skilled in the art should understand that the above-mentioned recognition in units of rows or columns is not limited to recognition in units of a single row or a single column, but may mean recognition in units of a fixed number of rows or columns. In this case, the patterns in the fixed number of rows or columns need to be put into the same binary tree.

[0095] In operation S205 , the central control unit 102 sequentially identifies patterns at respective nodes from each binary tree to obtain a target character sequence.

[0096] Exemplarily, when the central control unit 102 can construct a plurality of binary trees corresponding to each of the plurality of rows or columns, then each binary tree is identified in turn according to the order in which the binary trees are constructed.

[0097] Figure 3 The area 300 including the target character sequence and the corresponding binary trees are schematically shown.

[0098] Figure 3 The binary tree in the upper right portion of corresponds to the pattern in the left column in area 300, and the binary tree in the lower right portion corresponds to the pattern in the right column in area 300.

[0099] According to the two binary trees on the right and the two columns of patterns in area 300, the pattern placed at each node in the left subtree of the binary tree is a pattern before the reference pattern (e.g., 8 or 10 at the root node) placed at the root node of the binary tree, and the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located. The pattern placed at each node in the right subtree of the binary tree is a pattern after the reference pattern placed at the root node of the binary tree, and the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located.

[0100] Figure 4 The flowchart of determining a root node and determining the order of patterns to be traversed according to an embodiment of the present disclosure is schematically shown.

[0101] like Figure 4 As shown, the following operations are included.

[0102] In operation S401 , the central control unit 102 matches each pattern in the pattern set with characters in a database pre-stored in a memory, and determines a matching probability of each pattern.

[0103] In operation S403 , the central control unit 102 may determine the pattern whose matching probability is greater than a first predetermined threshold as a character.

[0104] In an exemplary embodiment, the first predetermined threshold may be 50%, 60%, 70%, 80%, 90% or 95%.

[0105] In an exemplary embodiment, the central control unit 102 may filter out patterns whose matching probability is less than a first predetermined threshold, and only retain patterns whose matching probability is greater than or equal to the first predetermined threshold.

[0106] In operation S405 , the central control unit 102 may form a character set using the characters determined in operation S403 .

[0107] In an exemplary embodiment, the central control unit 102 may also arrange the characters in the character set according to the matching probabilities of the characters.

[0108] In an exemplary embodiment, the central control unit 102 may arrange the characters in order of the matching probability from large to small.

[0109] In another exemplary embodiment, the central control unit 102 may also arrange the characters in ascending order.

[0110] In operation S407 , the central control unit 102 may use the character with the highest matching probability in the character set as the starting character, and traverse in descending order of matching probability.

[0111] Specifically, when the characters are arranged in a character set in descending order, the central control unit 102 can perform sequential traversal according to the arrangement order.

[0112] In the case where the characters are arranged in the character set in ascending order, the central control unit 102 needs to traverse in the reverse order of the arrangement.

[0113] In an exemplary embodiment, the central control unit 102 may select a starting character with a maximum matching probability and place it at the root node of the binary tree.

[0114] Furthermore, the central control unit 102 may remove the patterns that are not recognized as characters in the binary tree and retain only the characters, thereby forming a binary tree in which all nodes are characters.

[0115] Figure 5 Schematically illustrates an embodiment of the present disclosure Figure 3 The binary trees in are then each binary tree after removing the non-character patterns.

[0116] Figure 5 The upper binary tree in Figure 3 Corresponding to the binary tree on the upper right side of , Figure 5 The lower binary tree in Figure 3 It corresponds to the binary tree in the lower right part.

[0117] pass Figure 5 and Figure 3 From the binary tree in, we can see that Figure 5 Only characters are retained at each node in the binary tree, and the matching probability of character 10 at the root node is the maximum matching probability in the left column in area 300, and the matching probability of character 8 at the root node is the maximum matching probability in the right column in area 300.

[0118] Figure 6 The flowchart of placing characters in the same row or column into the same binary tree according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 6 As shown, the following operations are included.

[0120] In operation S601 , an angle between a line connecting the centroids of the traversed characters and the starting character and a predetermined axis is calculated.

[0121] In an exemplary embodiment, the predetermined axis may be a left or right side of a minimum rectangular bounding box surrounding an area including the target character sequence.

[0122] In an exemplary embodiment, the centroid of the traversed character may be the centroid of a minimum rectangular bounding box surrounding the traversed character.

[0123] In an exemplary embodiment, the centroid of the starting character may be the centroid of a minimum rectangular bounding box surrounding the starting character.

[0124] In operation S603, it is determined whether the calculated angle is less than or equal to a predetermined angle threshold.

[0125] In a case where the calculated angle is less than or equal to the predetermined angle threshold (“YES” in operation S603 ), the process proceeds to operation S605 .

[0126] In operation S605 , it is determined whether the traversed characters are in the same row or column as the starting character.

[0127] In operation S607, the characters determined to be in the same row or column are placed in a binary tree with the start character as the root node.

[0128] For example, Figure 3 The characters "8" and "5" in the left column of area 300 in Figure 3In the binary tree on the upper right side of , the character "8" is the starting character and is located at the root node of the binary tree. By calculating the angle between the centroid line of the characters "5" and "8" and the left side of the area 300, it is determined that the calculated angle is less than or equal to the predetermined angle threshold, and it is determined that the characters "5" and "8" are in the same column. Based on this, the character "5" is placed in the binary tree with the character "8" as the root node.

[0129] In a case where the calculated angle is greater than the predetermined angle threshold (“No” in operation S603 ), the process proceeds to operation S609 .

[0130] In operation S609 , it is determined whether the traversed character and the starting character are in different rows or columns.

[0131] In operation S611, the traversed characters are placed in a new binary tree different from the binary tree where the starting character is located.

[0132] Continuing with the above example, Figure 3 The character "10" in the right column of the region 300 in the image is determined by calculating the angle between the centroid line between the character "8" in the left column and the character "10" in the right column and the left side of the region 300, and determining that the calculated angle is greater than a predetermined angle threshold, and determining that the character "10" and the character "8" are in different columns. Based on this, the character "10" is placed in the right column. Figure 3 In another binary tree at the bottom of .

[0133] Furthermore, since the central control unit 102 traverses the characters in the order of matching probability from large to small, the character "10" is the first traversed character in the right column of the area 300. Figure 3 In a binary tree at the bottom of , it is placed at the root node.

[0134] Exemplarily, when placing characters in a plurality of rows or columns in the same binary tree, it may be determined whether the absolute value of the calculated angle is greater than a predetermined angle threshold.

[0135] Figure 7 A flowchart is schematically shown for further determining the position of the traversed character in the binary tree after determining that the traversed character is in the same row or column as the starting character according to an embodiment of the present disclosure.

[0136] like Figure 7 As shown, the following operations are included.

[0137] In operation S701, a first distance between a first point in a minimum rectangular bounding box surrounding a start character and a second point in a minimum rectangular bounding box surrounding an area including a target character sequence is calculated.

[0138] In operation S703, a distance between a third point in a minimum rectangular frame surrounding the traversed character and the second point is calculated.

[0139] In an exemplary embodiment, the position of the third point in the minimum rectangular bounding box surrounding the traversed characters corresponds to the position of the first point in the minimum rectangular bounding box surrounding the starting character.

[0140] Exemplarily, the first point in the minimum rectangular bounding box enclosing the starting character may be the point at the upper left corner of the minimum rectangular bounding box, and correspondingly, the third point in the minimum rectangular bounding box enclosing the traversed characters is also the point at the upper left corner of the minimum rectangular bounding box.

[0141] Exemplarily, the first point in the minimum rectangular bounding box enclosing the starting character may be the point at the upper right corner of the minimum rectangular bounding box, and correspondingly, the third point in the minimum rectangular bounding box enclosing the traversed characters is also the point at the upper right corner of the minimum rectangular bounding box.

[0142] Exemplarily, the first point in the minimum rectangular bounding box surrounding the starting character may be the centroid of the minimum rectangular bounding box, and correspondingly, the third point in the minimum rectangular bounding box surrounding the traversed characters is also the centroid of the minimum rectangular bounding box.

[0143] The selection of the first point and the third point above are exemplary rather than restrictive. Those skilled in the art should understand that any appropriate first point and third point can be selected as needed.

[0144] Exemplarily, the first point may be one of the following: a center point, an upper left corner point, a centroid point.

[0145] Exemplarily, the second point in the minimum rectangular bounding box enclosing the area including the target character sequence can be the upper left point, upper right point, lower left point, lower right point, centroid or any other suitable point in the minimum rectangular bounding box.

[0146] In operation S705 , it is determined whether the traversed character is before or after the starting character based on the first distance determined in operation S701 and the second distance determined in operation S703 .

[0147] For example, Figure 3 As shown in the area 300 and the two columns of patterns therein, when the point in the upper left corner or the upper right corner of the minimum rectangular border enclosing the area 300 is selected as the second point, when the second distance (for example, the distance from the center of mass of the character "5" to the point in the upper left corner) is greater than the first distance (for example, the distance from the center of mass of the character "8" to the point in the upper left corner), it is determined that the traversed character is located after the starting character.

[0148] Correspondingly, when the second distance (for example, the distance from the center of mass of the character "1" to the point in the upper left corner) is less than the first distance (for example, the distance from the center of mass of the character "8" to the point in the upper left corner), it is determined that the traversed character is located before the starting character.

[0149] For example, Figure 3 As shown in the area 300 and the two columns of patterns therein, when the point at the lower left corner or the lower right corner in the minimum rectangular border enclosing the area 300 is selected as the second point, when the second distance (for example, the distance from the center of mass of the character "5" to the point at the lower left corner) is less than the first distance (for example, the distance from the center of mass of the character "8" to the point at the lower left corner), it is determined that the traversed character is located after the starting character.

[0150] Correspondingly, when the second distance (for example, the distance from the center of mass of the character "1" to the point of the lower left corner) is greater than the first distance (for example, the distance from the center of mass of the character "8" to the point of the lower left corner), it is determined that the traversed character is located before the starting character.

[0151] Therefore, it can be determined whether the traversed character is located before or after the starting character based on the first distance and the second distance according to the relative positions of the selected first point, second point and third point.

[0152] Figure 8 The flowchart schematically shows the further processing when it is determined that the traversed characters and the starting character are in different rows or columns according to an embodiment of the present disclosure.

[0153] like Figure 8 As shown in , the following operations are included.

[0154] In the case where the region including the target character sequence includes multiple rows or columns, there are multiple corresponding predetermined angle thresholds θ i (where i=1, 2, . . . , N) At this time, it is determined within which range of the plurality of predetermined angle thresholds the angle calculated in operation S601 is located.

[0155] Specifically, in operation S801, it is determined that the calculated angle is greater than θ i and less than or equal to θ i+1 .

[0156] In operation S803, it is determined that the traversed character is in the (i+1)th row or column counted from the row or column where the start character is located.

[0157] In operation S805 , the traversed characters are placed in an i-th new binary tree that is different from the binary tree with the start character as the root node.

[0158] Exemplarily, the arrangement of characters in the new binary tree is the same as the arrangement of characters in the binary tree with the start character as the root node.

[0159] Exemplarily, since the characters are sequentially traversed in the order of matching probability from large to small, the character that is first determined to be in a new binary tree is placed at the root node. The characters after the character that are determined to be in the binary tree where the character is located are determined based on the method described above to determine whether the character should be in the left subtree or the right subtree.

[0160] Fig. 9 A block diagram of a binary tree-based character sorting device according to an embodiment of the present disclosure is schematically shown.

[0161] like Fig. 9 As shown, the binary tree-based character sorting device 900 may include a collection unit 1001 and a first processing unit 1002 .

[0162] The collection unit 1001 may be configured to collect patterns in a region including a target character sequence to obtain a pattern set.

[0163] The first processing unit 1002 may be configured to: traverse each pattern in the pattern set, place patterns in the same row or column in the same binary tree; and sequentially identify the pattern at each node in the binary tree to obtain a target character sequence.

[0164] When the first processing unit 1002 places the patterns in the same row or column in the same binary tree, the pattern placed at each node in the left subtree of the binary tree is a pattern before the reference pattern placed at the root node of the binary tree, and the degree to which the node where each character in the left subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located. In addition, the pattern placed at each node in the right subtree of the binary tree is a pattern after the reference pattern placed at the root node of the binary tree, and the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located.

[0165] In addition to the above modules 1001 and 1002 , the binary tree-based character sorting device 900 may further include other modules for correspondingly performing the above operations.

[0166] For the sake of clarity and brevity, each module and the corresponding operations performed therein are not described in detail here.

[0167] Based on the above binary tree-based character sorting method, an embodiment of the present application further provides an optical character sequence positioning and recognition method and a device for implementing the method. The optical character sequence positioning and recognition method may include: locating a region including a target character sequence in an input image; and using the above binary tree-based character sorting method to recognize the target character sequence in the region.

[0168] The optical character sequence positioning and recognition device may include: a positioning unit configured to locate a region including a target character sequence in an input image; and a second processing unit, using the above binary tree-based character sorting method to recognize the target character sequence in the region.

[0169] The optical character sequence positioning and recognition method and the device for implementing the method will be described below with reference to the accompanying drawings.

[0170] Optical character sequence positioning and recognition method and device for implementing the method

[0171] Fig.10 The flowchart of the optical character sequence positioning and recognition method according to the embodiment of the present disclosure is schematically shown.

[0172] like Fig.10 As shown, the following operations are included.

[0173] In operation S1001, upon inputting an image (eg, Figure 1 The input image 101 shown in FIG. 1 is used to locate a region including a target character sequence (eg, Figure 1 area A).

[0174] In operation S1003, a target character sequence is identified in the located area using the method according to the embodiment of the present application.

[0175] The operation of identifying the target character sequence in the located area using the character sorting method of the binary tree in operation S1003 is similar to the above operation and will not be described in detail here.

[0176] Here, the positioning operation in operation S1001 will be explained in detail.

[0177] When performing the positioning operation, one-level, two-level, or even multi-level area positioning is performed based on the characteristics of the target character sequence (e.g., color, texture, position distribution in the image), the size comparison between the image and character sequence areas, the complexity of the scene, etc.

[0178] Fig.11 A relatively simple scenario of character sequence region positioning according to an embodiment of the present disclosure is schematically shown.

[0179] Fig.11The container image is shown in Fig.11 It can be seen that the box number characters occupy a small part of the entire box body. The box number character sequence contains 11 or 15 characters, arranged horizontally, vertically, and tilted, including one to four rows or columns, but the character color is different from the background color, and the spatial distribution of the characters is relatively concentrated, which is significantly different from other non-character areas or non-box number character areas. Therefore, a simpler algorithm model can be used to directly locate the character sequence area.

[0180] In an exemplary embodiment, the input image size can be reduced, but the character sequence region still retains certain unique texture features and does not affect the recognition rate. Therefore, the computing resources and time used to locate a large number of character regions can be saved.

[0181] When there are multiple containers side by side in a large-size image, the size of the character sequence area is too different from the size of the entire image, and more features are lost after image compression / reduction, which easily leads to a decrease in recognition rate. In this case, the container body can be located first, and then the position of the container number character sequence can be located, so that the character sequence area can be located using two-level positioning.

[0182] Using the above positioning and using the binary tree-based character sorting method, we get Fig.12 The area including the target character sequence shown in ( Fig.12 ) and the final recognized target character sequence ( Fig.12 in the right figure).

[0183] Fig.13 The diagram schematically illustrates a relatively complex scenario of character sequence region positioning according to an embodiment of the present disclosure.

[0184] Fig.13 An image of a customs declaration is shown in FIG. Fig.13 As shown in the figure, the customs declaration image contains a lot of content, and there is a large amount of handwritten content. The customs declaration number area is a small target compared to the entire image, and it is difficult to locate the area directly from the image.

[0185] However, the position of the customs declaration number in the text is relatively fixed, and the customs declaration number is a sequence of small printed characters in a fixed format. Therefore, a multi-level positioning method can be used to perform layer-by-layer positioning using position information and texture information.

[0186] First, image preprocessing is used to determine the front and back of the scanned text image based on prior knowledge such as text texture features and text distribution, and to obtain the location of the box area where the customs declaration number is located in the front image, such as Fig.14 shown.

[0187] exist Fig.14In the first-level positioning frame shown in , a simple network model is used to obtain the positioning frame of the customs declaration number characters. The single-line customs declaration number character sequence basically fills the entire frame, and the result is as follows Fig.15 Then, the final target character sequence is identified in the secondary positioning frame using a binary tree-based character sorting method, such as Fig.15 As shown in the lower figure.

[0188] Fig.16 The program for implementing the above optical character sequence positioning and recognition method is schematically shown. Those skilled in the art should understand that Fig.16 The procedure shown in is only one implementation and does not have any limiting meaning. Fig.16 Certain instructions in the program may not be executed or may be replaced by other appropriate instructions, or Fig.16 The sequential drawing instructions in the example may be executed in parallel, simultaneously, in reverse order, or in any other suitable order, or may be executed in Fig.16 Add any appropriate directives in .

[0189] like Fig.16 As shown in , the input is a character category (e.g., character, non-character), a positioning box (e.g., a minimum rectangular border surrounding a pattern), a probability (e.g., the above-mentioned matching probability), an allowable angle error |Δ| (e.g., the above-mentioned predetermined angle threshold), and the output is a character sequence containing multiple binary trees.

[0190] implement Fig.16 The operations that can be implemented by the instructions shown in have been described in detail above and will not be repeated here.

[0191] Fig.17 A block diagram of an optical character sequence positioning and recognition device is schematically shown.

[0192] like Fig.17 As shown, the optical character sequence positioning and recognition device 1700 may include a positioning unit 1701 and a second processing unit 1702 .

[0193] The positioning unit 1701 may be configured to locate a region including a target character sequence in the input image.

[0194] The number of positioning units 1701 can be multiple according to the complexity of the scene.

[0195] The second processing unit 1702 may be configured to identify a target character sequence in the region using a character sorting method based on a binary tree.

[0196] In addition to the above modules 1701 and 1702, the optical character sequence positioning and recognition device 1700 may also include other modules for correspondingly performing the above operations.

[0197] For the sake of clarity and brevity, each module and the corresponding operations performed therein are not described in detail here.

[0198] The functions of multiple modules according to the embodiments of the present disclosure can be implemented in one module. A module according to the embodiments of the present disclosure can be split into multiple modules for implementation. The module according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, the module according to the embodiments of the present disclosure can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0199] According to an embodiment of the present disclosure, at least one of the above modules may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Optionally, at least one of the above modules may be at least partially implemented as a computer program module, which may perform corresponding functions when the computer program module is executed.

[0200] Fig.18 A block diagram of an electronic device suitable for implementing a binary tree-based character sorting method and an optical character sequence positioning and recognition method according to an embodiment of the present disclosure is schematically shown.

[0201] Fig.18 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0202] like Fig.18As shown, the electronic device 1800 according to an embodiment of the present disclosure includes a processor 1801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1802 or a program loaded from a storage part 1808 to a random access memory (RAM) 1803. The processor 1801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1801 may also include an onboard memory for caching purposes. The processor 1801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0203] In RAM 1803, various programs and data required for the operation of electronic device 1800 are stored. Processor 1801, ROM 1802 and RAM 1803 are connected to each other via bus 1804. Processor 1801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1802 and / or RAM 1803. It should be noted that the program can also be stored in one or more memories other than ROM 1802 and RAM 1803. Processor 1801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0204] According to an embodiment of the present disclosure, the electronic device 1800 may further include an input / output (I / O) interface 1805, which is also connected to the bus 1804. The electronic device 1800 may further include one or more of the following components connected to the I / O interface 1805: an input portion 1806 including a keyboard, a mouse, etc.; an output portion 1807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1808 including a hard disk, etc.; and a communication portion 1809 including a network interface card such as a LAN card, a modem, etc. The communication portion 1809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 1805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 1808 as needed.

[0205] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1809, and / or installed from the removable medium 811. When the computer program is executed by the processor 1801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0206] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0207] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1802 and / or RAM 1803 described above and / or one or more memories other than ROM 1802 and RAM 1803.

[0208] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0209] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0210] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A character sorting method based on a binary tree, comprising: Performing pattern collection in a region including the target character sequence to obtain a pattern set; Traverse each pattern in the pattern set, place the patterns in the same row or column in the same binary tree, and Sequentially identify the patterns at each node from the binary tree to obtain the target character sequence, Wherein, traversing each pattern in the pattern set includes: Determining the probability that each pattern in the pattern set matches a character pre-stored in a database; Determine the pattern with a probability greater than a first predetermined threshold as a character; forming a character set using the determined characters; and The character with the highest probability in the character set is used as the starting character, and each character in the character set is traversed in descending order of probability.

2. The binary tree-based character sorting method according to claim 1, wherein: The placing of patterns in the same row or column into the same binary tree comprises: The pattern placed at each node in the left subtree of the binary tree is a pattern placed before the reference pattern at the root node of the binary tree, and wherein the degree of distance between the node where each character in the left subtree is located and the root node corresponds to the degree of distance between the pattern at the node and the reference pattern in the row or column where the node is located, and The pattern placed at each node in the right subtree of the binary tree is a pattern placed after the reference pattern at the root node of the binary tree, and wherein the degree to which the node where each pattern in the right subtree is located is away from the root node corresponds to the degree to which the pattern at the node is away from the reference pattern in the row or column where it is located.

3. The character sorting method based on binary tree according to claim 1, wherein: The patterns placed at the nodes in the binary tree are all characters, and the reference pattern placed at the root node in the binary tree is the starting character.

4. According to the binary tree-based character sorting method of claim 3, placing characters in the same row or column in the same binary tree comprises: Calculate the angle between the line connecting the centroids of the traversed character and the starting character and the predetermined axis; In response to the calculated angle being less than or equal to a predetermined angle threshold, determining that the traversed character is in the same row or column as the starting character, and placing the traversed character in a binary tree with the starting character as a root node; as well as In response to the calculated angle being greater than the predetermined angle threshold, it is determined that the traversed character is in a different row or column from the starting character, and the traversed character is placed in a new binary tree.

5. According to the binary tree-based character sorting method of claim 4, after determining that the traversed character is in the same row or column as the starting character, the following operation is further performed: Calculate a first distance between a first point in a minimum rectangular frame surrounding the starting character and a second point in a minimum rectangular frame surrounding the area; Calculating a second distance between a third point in the minimum rectangular frame surrounding the traversed characters and the second point, wherein a position of the third point in the minimum rectangular frame surrounding the traversed characters corresponds to a position of the first point in the minimum rectangular frame surrounding the starting character; and Based on the first distance and the second distance, it is determined whether the traversed character is before or after the starting character.

6. According to the binary tree-based character sorting method of claim 4, in response to the calculated angle being greater than the predetermined angle threshold, determining that the traversed character is in a different row or column from the starting character comprises: In the case where there are multiple rows or columns in the region, there are multiple corresponding predetermined angle thresholds θ i , where i = {1, 2, ..., N}, and where, when the calculated angle is greater than θ i and is less than or equal to θ i+1 , it is determined that the traversed character is in the i+1th row starting from the row or column where the starting character is located, and the traversed character is correspondingly placed in the i-th new binary tree.

7. The binary tree-based character sorting method according to claim 4, wherein: The arrangement of characters in the new binary tree is the same as the arrangement of characters in the binary tree with the starting character as the root node.

8. The binary tree-based character sorting method according to claim 1, wherein: The first point is one of the following: a center point, an upper left corner point, a centroid point; and The second point is one of the following: an upper left corner point, an upper right corner point, a lower left corner point, or a lower right corner point.

9. A character sorting device based on a binary tree, comprising: A collection unit, configured to collect patterns in a region including a target character sequence to obtain a pattern set; as well as The first processing unit is configured as follows: Traversing each pattern in the pattern set, and placing patterns in the same row or column into the same binary tree; as well as Sequentially identify the patterns at each node from the binary tree to obtain the target character sequence, Wherein, the first processing unit is further configured to: Determining the probability that each pattern in the pattern set matches a character pre-stored in a database; Determine the pattern with a probability greater than a first predetermined threshold as a character; forming a character set using the determined characters; and The character with the highest probability in the character set is used as the starting character, and each character in the character set is traversed in descending order of probability.

10. A method for locating and recognizing an optical character sequence, comprising: Locating a region in an input image that includes a target character sequence; as well as The target character sequence is identified in the region by using the binary tree-based character sorting method according to any one of claims 1 to 8.

11. The optical character sequence positioning and recognition method according to claim 10, further comprising: When performing the positioning operation, at least one level of area positioning is performed based on the characteristics of the target character sequence, the size comparison between the input image and the area including the target character sequence, and the complexity of the scene.

12. An optical character sequence positioning and recognition device, comprising: A positioning unit configured to locate a region including a target character sequence in an input image; as well as The second processing unit identifies the target character sequence in the region by using the binary tree-based character sorting method according to any one of claims 1 to 8.

13. An electronic device, comprising: one or more processors; as well as a memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the binary tree-based character sorting method according to any one of claims 1 to 8 or the optical character sequence positioning and recognition method according to claim 10.

14. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the binary tree-based character sorting method according to any one of claims 1 to 8 or the optical character sequence positioning and recognition method according to claim 10.

Images