Text recognition method and related apparatus

Abstract

The application discloses a text recognition method and related device, wherein the method comprises: calculating a rotation angle of each text recognition box, the text recognition box comprising at least a group of semantically coherent texts; performing DBScan clustering on the rotation angles of a plurality of text recognition boxes to obtain a plurality of clustering clusters; separating text recognition boxes that do not meet requirements in each clustering cluster and arranging text recognition boxes that meet requirements in each clustering cluster according to a preset word order to obtain a target word order arrangement group; and matching the target word order arrangement group with commodity labels in a preset dictionary library to obtain a recognition result. The application can be used for a single picture of intensive marketing text and has good recognition effect.

Description

Technical Field

[0001] This application relates to the field of text recognition technology, and in particular to text recognition methods and related devices. Background Technology

[0002] Leading brands in the traditional marketing field are accelerating the iteration of their products and seeking material management platforms to store and manage their various product SKUs. Manual tagging is too labor-intensive, so they use various auxiliary methods, such as configuring dictionary libraries to identify and match all text information in the image text before tagging the image, completing a large amount of repetitive tagging work. However, marketing materials often have their original text information segmented due to layout and promotional reasons, using different font sizes and rotation angles, which increases the computational complexity of the matching process and requires more matching rules.

[0003] The existing text information is not broken down but exists as a complete sentence. The required information can be extracted from this complete sentence. The existing text exists in a structured and regular form in the table. The information of different units in the table can be directly identified through models or rules.

[0004] In the marketing field, there are special cases in product posters: Due to image editing, text is broken down into different segments and exists in the same line. For example, "Vibrant and Fresh Limited Edition" is broken down into "Vibrant" and "Fresh Limited Edition". Furthermore, for visual emphasis, text within the same sentence may be broken down into different modules of varying sizes. For instance, "Perfect Concealer Foundation" in an image is broken down into "Perfect", "Concealer", and "Foundation", with "Perfect" being significantly larger than "Concealer". For aesthetically pleasing layout, different texts may surround the product image at different angles; for example, "Vibrant and Fresh Limited Edition" and "Perfect Concealer Foundation" may be distributed at different angles and in different positions.

[0005] In related technologies, if it is necessary to label an image based on information in the image, the image label is the parent product label: "Perfect Concealer Foundation," and the child product label: "Vibrant and Fresh Limited Edition." To identify these labels, it is necessary to arrange and combine all OCR text sub-units in the image, and then perform text information matching on the different long text information arranged.

[0006] In practice, this method requires permutations and combinations because the machine does not know the semantic information or the length of the text to be matched in the image. Each combination requires an additional matching attempt. This method increases significantly as the number of words recognized in the image increases. For example, if a full permutation is performed on 3 words, there may be 6 possible cases. If a full permutation is performed on 10 words, there may be 3,628,800 possible cases.

[0007] There is currently no effective solution to the problem of low efficiency in text recognition and matching in related technologies. Summary of the Invention

[0008] The main purpose of this application is to provide a text recognition method and related apparatus to address the shortcomings of related technologies.

[0009] To achieve the above objectives, according to one aspect of this application, a text recognition method is provided.

[0010] The text recognition method according to this application includes:

[0011] Calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least one semantically coherent set of text;

[0012] The rotation angles of the multiple text recognition boxes are clustered using DBScan to obtain multiple clusters;

[0013] Separate the non-compliant text recognition boxes within each cluster and arrange the compliant text recognition boxes within each cluster according to a preset word order to obtain the target word order arrangement group;

[0014] The target word order group is matched with product tags in a preset dictionary to obtain the recognition result.

[0015] In some embodiments, separating non-compliant text recognition boxes within each cluster includes:

[0016] Calculate the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster;

[0017] Based on the normalization threshold, after separating the text recognition boxes that do not meet the requirements, the text in the remaining clusters is reclassified to obtain the minimum clusters. Each cluster includes at least two minimum clusters and each minimum cluster includes at least one text recognition box.

[0018] In some embodiments, arranging the text recognition boxes that meet the requirements within each cluster according to a preset word order to obtain a target word order arrangement group includes:

[0019] Using text reading habits as prior knowledge, DBScan clusters the text recognition boxes that meet the requirements within each cluster in a preset coordinate system;

[0020] Based on the DBScan clustering results, determine the positional and categorical relationships between the text recognition boxes that meet the requirements within each cluster. The positional relationships include vertical and horizontal relationships, and the categorical relationships include distance relationships.

[0021] After arranging the text recognition boxes that meet the requirements in each cluster according to the positional relationship and the category relationship, the target word order arrangement group of the text recognition boxes is obtained.

[0022] In some embodiments, calculating the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster includes:

[0023] The average angle within the same cluster is used as the coordinate system transformation angle within that cluster.

[0024] Map the rotation angle of each text recognition box within the same cluster to a preset coordinate system;

[0025] Based on the preset coordinate system, the normalized threshold between each pair of text recognition boxes is calculated to determine whether the pairs of text recognition boxes can be spliced ​​together.

[0026] In some embodiments, the step of performing DBScan clustering on the rotation angles of the multiple text recognition boxes to obtain multiple clusters includes:

[0027] Determine whether the difference in rotation angle between any two text recognition boxes is greater than a preset threshold;

[0028] If it is greater than, then do not connect;

[0029] If the value is less than 1, the two text recognition boxes are connected to obtain multiple clusters, and each cluster includes at least one text recognition box.

[0030] In some embodiments, the calculation of the rotation angle of each text recognition box, wherein the text recognition box comprises at least a set of semantically coherent text, further includes:

[0031] Obtain images containing at least the text information of the marketing poster packaging;

[0032] Based on OCR, semantically coherent text in the text information image is labeled as a text recognition box, and the coordinate information of the four corner points of each text recognition box is determined.

[0033] In some embodiments, calculating the rotation angle of each text recognition box includes:

[0034] Based on the coordinate information of the four corner points of each text recognition box, calculate the vector corresponding to the preset corner point;

[0035] The angle between the vector and the unit vector of the preset coordinate axis is obtained by the cross product between the different vectors.

[0036] To achieve the above objectives, according to another aspect of this application, a text recognition device is provided.

[0037] The text recognition device according to this application includes:

[0038] A calculation module is used to calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text;

[0039] The clustering module is used to perform DBScan clustering on the rotation angles of multiple text recognition boxes to obtain multiple clusters;

[0040] The arrangement module is used to separate the non-compliant text recognition boxes in each cluster and arrange the compliant text recognition boxes in each cluster according to a preset word order to obtain a target word order arrangement group.

[0041] A matching module is used to match the target word order arrangement group with product tags in a preset dictionary to obtain a recognition result. According to another aspect of the present invention, a storage medium is also provided, wherein the storage medium stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments at runtime.

[0042] According to another aspect of the invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0043] In the embodiments of this application, the text recognition method and related apparatus employ a technical solution that calculates the rotation angle of each text recognition box and performs DBScan clustering on the rotation angles of multiple text recognition boxes to obtain multiple clusters. This achieves the goal of separating the non-compliant text recognition boxes within each cluster and arranging the compliant text recognition boxes within each cluster according to a preset word order to obtain a target word order arrangement group. This enables the matching of the target word order arrangement group with product tags in a preset dictionary to obtain recognition results, thereby solving the technical problem of low text recognition matching efficiency in related technologies. Attached Figure Description

[0044] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0045] Figure 1 This is a flowchart illustrating a text recognition method according to an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of a text recognition box in a text recognition method according to an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the rotation angle of the text recognition box in the text recognition method according to an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the DBScan clustering principle in the text recognition method according to the embodiments of this application;

[0049] Figure 5 This is a schematic diagram of clustering in the text recognition method according to an embodiment of this application;

[0050] Figure 6 This is a schematic diagram illustrating the principle of sorting text recognition boxes in the text recognition method according to the embodiments of this application;

[0051] Figure 7 This is a schematic diagram of the structure of a text recognition device according to an embodiment of this application; Detailed Implementation

[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0053] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0054] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0055] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0056] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linking," and "socketing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0057] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0058] During their research, the inventors discovered that in posters using related technologies, text with a certain semantic coherence is often grouped together by designers, even when using different font sizes. A key approach is to prioritize clustering this poster information and then sorting it according to reading comprehension habits, significantly improving matching efficiency. For example, if there are 10 words in an image, clustering and sorting can directly yield 4 semantically consistent "long texts." Generally, semantic matching of these 4 long texts is sufficient. If combining information is required, these 4 long texts can be arranged and combined for matching.

[0059] Therefore, the text recognition method in this application adopts a new algorithm for text matching of marketing materials extracted in this scenario. After text extraction, it prioritizes clustering into texts with word order that conforms to certain reading habits based on the relative position and size of the text, as well as the rotation angle, and then performs text matching and tagging, thereby improving tagging efficiency and simplifying the tagging work.

[0060] like Figure 2 As shown, the method includes the following steps S110 to S140:

[0061] Step S110: Calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text.

[0062] First, based on OCR, the text in the image is recognized and bounding boxes are drawn to obtain the text recognition boxes, and the four coordinates of all text recognition boxes are given. Then, the rotation angle is calculated for each text recognition box.

[0063] Step S120: The rotation angles of the multiple text recognition boxes are clustered using DBScan to obtain multiple clusters.

[0064] After calculating the angles of all text boxes, clustering is performed using the DBScan algorithm.

[0065] It's understandable that DBScan uses a clustering approach, meaning that as long as the distance between two points is within a set range, they are connected, and finally all points that meet the conditions are connected to form a cluster.

[0066] Step S130: Separate the non-compliant text recognition boxes within each cluster and arrange the compliant text recognition boxes within each cluster according to a preset word order to obtain the target word order arrangement group.

[0067] After clustering is completed, the main tasks are to separate texts with excessively large intervals within the cluster and sort texts within the cluster according to the word order that meet the interval requirements.

[0068] Step S140: Match the target word order group with the product tags in the preset dictionary to obtain the recognition result.

[0069] The sorted and combined text is matched with the known SKU descriptions of products in the preset dictionary to obtain the final matching result.

[0070] As can be seen from the above description, this application achieves the following technical effects:

[0071] In the embodiments of this application, the text recognition method and related apparatus employ a technical solution that calculates the rotation angle of each text recognition box and performs DBScan clustering on the rotation angles of multiple text recognition boxes to obtain multiple clusters. This achieves the goal of separating the non-compliant text recognition boxes within each cluster and arranging the compliant text recognition boxes within each cluster according to a preset word order to obtain a target word order arrangement group. This enables the matching of the target word order arrangement group with product tags in a preset dictionary to obtain recognition results, thereby solving the technical problem of low text recognition matching efficiency in related technologies.

[0072] In a preferred embodiment, separating the non-compliant text recognition boxes within each cluster includes: calculating a normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster; and, based on the normalization threshold, separating the non-compliant text recognition boxes and then reclassifying the remaining text within the clusters to obtain a minimum cluster, wherein each cluster includes at least two minimum clusters and each minimum cluster includes at least one text recognition box.

[0073] In practice, to separate the non-compliant text recognition boxes within each cluster, a normalization threshold can be calculated between each pair of text recognition boxes within the cluster. Based on the normalization threshold, the non-compliant text recognition boxes are separated, and the remaining text within the cluster is reclassified to obtain the smallest cluster.

[0074] It is important to note that, to meet the clustering requirements, each cluster must include at least two minimum clusters. Furthermore, each minimum cluster must contain at least one text recognition box. In this way, the size of the box does not need to be considered, allowing for further classification of the text within a cluster.

[0075] For example, the elements in cluster 3: ("gift bag", "promotional pack", "foundation", "concealer", "perfect") can be further divided into cluster 3_a ("gift bag", "promotional pack") and cluster 3_b ("foundation", "concealer", "perfect").

[0076] In a preferred embodiment, arranging the text recognition boxes that meet the requirements within each cluster according to a preset word order to obtain a target word order arrangement group includes: using text reading habits as prior knowledge, performing DBScan clustering on the text recognition boxes that meet the requirements within each cluster in a preset coordinate system; determining the positional and categorical relationships between the text recognition boxes that meet the requirements within each cluster based on the DBScan clustering results, wherein the positional relationships include vertical and horizontal relationships, and the categorical relationships include distance relationships; and arranging the text recognition boxes that meet the requirements within each cluster according to the positional and categorical relationships to obtain the target word order arrangement group of the text recognition boxes.

[0077] In specific implementation, by taking text reading habits as prior knowledge, the text recognition boxes that meet the requirements in each cluster are clustered by DBScan in a preset coordinate system. Based on the DBScan clustering results, the positional relationship and category relationship between the text recognition boxes that meet the requirements in each cluster are determined. Finally, the text recognition boxes that meet the requirements in each cluster are arranged in a preset word order according to the positional relationship and the category relationship to obtain the target word order arrangement group of the text recognition boxes.

[0078] In a preferred embodiment, the step of calculating the normalization threshold between pairs of text recognition boxes based on each text recognition box within the cluster includes: using the average angle within the same cluster as the coordinate system transformation angle within that cluster; mapping the rotation angle of each text recognition box within the same cluster to a preset coordinate system; and calculating the normalization threshold between pairs of text recognition boxes based on the preset coordinate system to determine whether the pairs of text recognition boxes can be spliced ​​together.

[0079] In practice, suppose a cluster contains four text boxes with center point coordinates (x1, y1), (x2, y2), (x3, y3), and (x4, y4). The rotation angles are calculated from the rotation angles of the text boxes, and are a1, a2, a3, and a4, respectively. Since even a difference of 1 or 2 degrees can significantly affect the final result during coordinate transformation, we average the angles of the text boxes within the same cluster. The angle A = (a1 + a2 + a3 + a4) / 4. This is the coordinate system transformation angle within the cluster. In other words, the average angle within the same cluster is used as the coordinate system transformation angle for that cluster.

[0080] Based on the preset coordinate system, the normalization threshold between each pair of text recognition boxes is calculated to determine whether the two pairs of text recognition boxes can be spliced. Then, the original coordinates need to be mapped to a new coordinate system through angle A. Next, the normalization threshold is used to determine whether the two boxes should be connected.

[0081] In a preferred embodiment, the step of performing DBScan clustering on the rotation angles of the multiple text recognition boxes to obtain multiple clusters includes: determining whether the difference in rotation angle between any two text recognition boxes is greater than a preset threshold; if it is greater, then they are not connected; if it is less, then the two text recognition boxes are connected to obtain multiple clusters, wherein each cluster includes at least one text recognition box.

[0082] In practice, this method eliminates the need to consider the size of the boxes, allowing text within a cluster to be reclassified.

[0083] As a preferred embodiment, the step of calculating the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text, further includes: obtaining a text information image that includes at least marketing poster packaging information; labeling the semantically coherent text in the text information image as a text recognition box based on OCR, and determining the coordinate information of the four corner points of each text recognition box.

[0084] In practice, a text information image containing at least marketing poster packaging information is obtained. Then, based on OCR, the semantically coherent text within the text information image is labeled as a text recognition bounding box, i.e., as shown below. Figure 2 As shown, the terms include "base color", "repair", "efficacy", "vitality", "energy", "perfect", "concealing", "fresh limited edition", "foundation", "promotional pack", and "gift bag".

[0085] As a preferred embodiment, the calculation of the rotation angle of each text recognition box includes: calculating the vector corresponding to the preset corner point based on the coordinate information of the four corner points of each text recognition box; and obtaining the angle between the vector and the preset coordinate axis unit vector based on the cross product between different vectors.

[0086] In practice, the rotation angle is calculated for each text box. For example, by calculating the cross product between AB, CD and the unit vector (1, 0), the angle between AB, CD and the X-axis unit vector can be obtained.

[0087] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0088] According to embodiments of this application, a text recognition device for implementing the above method is also provided, such as... Figure 7 As shown, the device includes:

[0089] The calculation module 710 is used to calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text;

[0090] Clustering module 720 is used to perform DBScan clustering on the rotation angles of multiple text recognition boxes to obtain multiple clusters;

[0091] The arrangement module 730 is used to separate the non-compliant text recognition boxes in each cluster and arrange the compliant text recognition boxes in each cluster according to a preset word order to obtain a target word order arrangement group.

[0092] The matching module 740 is used to match the target word order group with product tags in a preset dictionary to obtain the recognition result.

[0093] In this embodiment, the calculation module 710 first uses OCR to recognize and draw text frames in the image, obtaining the text recognition frames and providing the four coordinates of all text recognition frames. Then, it calculates the rotation angle for each text recognition frame.

[0094] In the clustering module 720 of this application embodiment, after calculating the angle of all text boxes, clustering is performed using the DBScan algorithm.

[0095] It's understandable that DBScan uses a clustering approach, meaning that as long as the distance between two points is within a set range, they are connected, and finally all points that meet the conditions are connected to form a cluster.

[0096] After clustering is completed in the arrangement module 730 of this application embodiment, the main functions are: to separate text with excessively large intervals within a cluster and to sort text within a cluster according to the word order that meets the interval requirements.

[0097] In this embodiment, the matching module 740 matches the sorted and combined text with the known product SKU descriptions in the preset dictionary to obtain the final matching result.

[0098] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0099] To better understand the above text recognition method process, the following explanation of the above technical solution is provided in conjunction with preferred embodiments, but it is not intended to limit the technical solution of the embodiments of the present invention.

[0100] Step S1, the recognition result is as follows Figure 1 As shown: “Base Color”, “Repair”, “Efficacy”, “Vibration”, “Energy”, “Perfect”, “Concealer”, “Fresh Limited Edition”, “Foundation”, “Promotional Pack”, “Gift Bag Included”.

[0101] Step S2: Calculate the rotation angle for each text box. This is done by subtracting the coordinates of the top-left corner from the coordinates of the top-right corner. For example, vector AB = (x2-x1, y2-y1), vector CD = (x4-x3, y4-y3). By calculating the cross product of AB, CD, and the unit vector (1, 0), the angle between AB, CD, and the unit vector of the X-axis can be obtained. AB is 30 degrees, and CD is -60 degrees. Figure 3 As shown.

[0102] Step S3: After calculating the angle of all text boxes, cluster them using the DBScan algorithm.

[0103] like Figure 4 As shown, DBScan uses a clustering approach, meaning that as long as the distance between two points is within a set distance range, they are connected. Ultimately, all points that meet the conditions are connected to form a cluster. For example, in the figure, the distance between x1 and x2 is less than the threshold, so they can be connected. Similarly, x2 can be connected to x3. After connecting all the points, when it is found that x3 can be connected to x4, they can belong to the same class. At the same time, another parameter is set: as long as the number of points in a cluster is greater than a certain number, such as 2, that is, as long as there are at least two points in a cluster, the cluster is considered valid (Note: here x1 is not the aforementioned coordinate, but is used to represent a point).

[0104] Based on this algorithm, angle calculations are incorporated, and a threshold, such as 5 degrees, is set. If the rotation angles differ by less than 5 degrees, the text boxes are considered to meet the condition. For example, in the image, the rotation angles of "perfect," "concealer," and "foundation" are 28 degrees, 25 degrees, and 22 degrees respectively. Therefore, "perfect" can be connected to "concealer," and "concealer" can be connected to "foundation." Meanwhile, the rotation angle of "vitality" is -20 degrees, which does not meet the 5-degree connection threshold with these three words, so it cannot be included. Ultimately, through this method, in... Figure 5 Four clusters were obtained.

[0105] Cluster 1: ("Efficacy", "Repair", "Background Color")

[0106] Cluster 2: ("Fresh Limited Edition", "Vibrant", "Energetic")

[0107] Cluster 3: ("Gift bag", "Promotional pack", "Foundation", "Concealer", "Perfect")

[0108] Step S4: After clustering is completed, two steps are performed to separate texts with excessively large intervals within a cluster and sort texts within the cluster that meet the interval requirements by word order.

[0109] Whether determining whether text within a cluster should be merged or separated, or the connection order of text within a cluster, coordinate transformation is required. This is because the current coordinate system is a Cartesian coordinate system with the top-left corner of the image as the origin. Using this coordinate system for judgments leads to inconsistent rules.

[0110] Reason 1 for coordinate transformation: For example, for "Vitality" and "Energy" above, the order should be "Vitality" first, then "Energy". At this time, "Energy" is in the upper right direction of "Vitality". However, for "Perfect" and "Concealer", the order should be "Perfect" first, then "Concealer". At this time, "Concealer" is in the lower right corner of "Perfect". There is no way to unify the rules here.

[0111] Reason 2 for coordinate transformation: The distance between text boxes can be clustered again using the DBscan approach. The words "perfect", "concealer", and "foundation" are relatively close in cluster 3, while "promotional pack" and "gift bag" are relatively far in cluster 3. However, this distance is relative and is affected by text size and text rotation angle. For example, the human eye can judge that "perfect" and "promotional pack" should not be connected. The judgment is not based on distance, but on the horizontal and vertical component distances in the tilt direction of the text. Although the difference in their horizontal component distances in the tilt direction is small, the difference in their horizontal component distances in the tilt direction is large, so they are not connected.

[0112] like Figure 6 As shown, suppose there are 4 text boxes in a cluster, and the center point coordinates of the 4 text boxes are (x1,y1), (x2,y2), (x3,y3), and (x4,y4). The rotation angles are calculated from the rotation angles of the text boxes, and are a1, a2, a3, and a4 respectively. Since the angle difference in coordinate transformation can be as small as 1 or 2 degrees, it will have a significant impact on the final result. Therefore, the angles of the text boxes in the same cluster are averaged here. Angle A = (a1 + a2 + a3 + a4) / 4. This is the angle of coordinate system transformation within the cluster.

[0113] Next, the original coordinates need to be mapped to the new coordinate system through angle A. The formula is as follows:

[0114] x' = x * cosA + y * sinA ◦ y' = -x * sinA + y * cos A ◦

[0115] In this way, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are converted into (x1', y1'), (x2', y2'), (x3', y3'), (x4', y4') in the new coordinate system.

[0116] Next, a normalization threshold is used to determine whether to connect two boxes. For Box 1 and Box 2:

[0117] As shown in the figure, delta x_12 = | x1' - x2'|

[0118] As shown in the figure, delta y_12 = | y1' - y2'|

[0119] The threshold is threshold x_12 = (Width 1 + Width 2) / 2 * 1.3

[0120] threshold y_12 = (Height 1 + Height 2) / 2 * 1.3

[0121] Only when delta x_12 < threshold x_12 and delta y_12 < threshold y_12 are both satisfied can it be determined that Box 1 and Box 2 are connected. For example, the reason why Box 3 and Box 4 cannot be connected is that delta y_34 > threshold y_34. / / 这里的 和

[0122] 在原文中似乎没有实际意义,翻译时保留原样

[0122] In this way, regardless of the size of the boxes, the text within a cluster can be classified again.

[0123] For example, the elements in Cluster 3: ("Gift bag", "Promotional pack", "Foundation", "Concealer", "Perfect") are further divided into Cluster 3_a ("Gift bag", "Promotional pack") and Cluster 3_b ("Foundation", "Concealer", "Perfect").

[0124] In the last step, within the smallest cluster, the order is determined. For example, for Cluster 3_b ("Foundation", "Concealer", "Perfect"), how to become the ordered text "Perfect Concealer Foundation". Among them, "Foundation", "Concealer", "Perfect" correspond to Box 1, Box 2, and Box 3 in the following figure respectively.

[0125] Prior knowledge indicates that reading habits are generally from top to bottom and from left to right. Based on this experience, first, perform DBScan clustering on the y-coordinates y1', y2', and y3' of boxes 1, 2, and 3 in the new coordinate system X'-Y'. It is found that the y-coordinates of box 1 and box 2 are approximately the same and should be classified into one category, while the coordinates of box 3 should be classified into a separate category. At the same time, the average value of the coordinates of boxes 1 and 2, (y1' + y2') / 2, is smaller than the y3' value of box 3, indicating that the text in boxes 1 and 2 should be above box 3. Then, compare the x1' and x2' of boxes 1 and 2, and it is found that x1' < x2'. So, box 1 should be on the left side of box 2.

[0126] Based on the above experience, the arrangement order of the three boxes is obtained as box 1, box 2, and box 3. In this order, connect the text in the boxes to get the final order "Perfect Concealer Foundation".

[0127] Step S5: Match the sorted and combined text with the known product sku descriptions in the preset dictionary library to obtain the final result.

[0128] Through the above process, the machine knows that the text combination order in the image is: "Base Color Repair Effect", "Vitality Fresh Limited Edition", "Perfect Concealer Foundation", "Promotional Set with Gift Bag".

[0129] By matching the existing labels in the dictionary library, it is known that the main label of the sku of this product is "Perfect Concealer Foundation", and the sub-label is "Vitality Fresh Limited Edition".

[0130] After the final effect verification, for a single picture of intensive marketing text, the average matching time is reduced from the original about 10 seconds to 0.6 seconds. The contrast improvement in efficiency will be more obvious as the number of marketing copy in the picture increases.

[0131] Due to the normalization principle of threshold determination, for adjacent texts of different sizes and different rotation angles, the normalized threshold can be calculated batchwise. In this way, the size of the threshold automatically adapts to the size of the text font to determine whether texts need to be spliced, avoiding the bad cases introduced in the case of a fixed threshold.

[0132] Since the scattered texts in the spliced pictures can be sorted according to a certain editing semantics, and at the same time, some long text label matches are supported, avoiding the incompatibility of regular texts.

[0133] The embodiment of the present application also provides a storage medium, in which a computer program is stored. Wherein, the computer program is set to execute the steps in any one of the above method embodiments when running.

[0134] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0135] S1, calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text;

[0136] S2, perform DBScan clustering on the rotation angles of the multiple text recognition boxes to obtain multiple clusters;

[0137] S3, separate the non-compliant text recognition boxes in each cluster and arrange the compliant text recognition boxes in each cluster according to a preset word order to obtain the target word order arrangement group;

[0138] S4. Match the target word order group with the product tags in the preset dictionary to obtain the recognition result.

[0139] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0140] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0141] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0142] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0143] S1, calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text;

[0144] S2, perform DBScan clustering on the rotation angles of the multiple text recognition boxes to obtain multiple clusters;

[0145] S3, separate the non-compliant text recognition boxes in each cluster and arrange the compliant text recognition boxes in each cluster according to a preset word order to obtain the target word order arrangement group;

[0146] S4. Match the target word order group with the product tags in the preset dictionary to obtain the recognition result.

[0147] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0148] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A text recognition method, characterized in that, The method includes: Calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least one semantically coherent set of text; The rotation angles of the multiple text recognition boxes are clustered using DBScan to obtain multiple clusters; Separate the non-compliant text recognition boxes within each cluster and arrange the compliant text recognition boxes within each cluster according to a preset word order to obtain the target word order arrangement group; The target word order arrangement group is matched with product tags in a preset dictionary to obtain the recognition result. The step of separating non-compliant text recognition boxes within each cluster includes: Calculate the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster; Based on the normalization threshold, after separating the text recognition boxes that do not meet the requirements, the remaining text within the clusters is reclassified to obtain the minimum clusters. Each cluster includes at least two minimum clusters, and each minimum cluster includes at least one text recognition box. The text recognition boxes that meet the requirements within each cluster are arranged according to a preset word order to obtain a target word order arrangement group, including: Using text reading habits as prior knowledge, DBScan clusters the text recognition boxes that meet the requirements within each cluster in a preset coordinate system; Based on the DBScan clustering results, determine the positional and categorical relationships between the text recognition boxes that meet the requirements within each cluster. The positional relationships include vertical and horizontal relationships, and the categorical relationships include distance relationships. After arranging the text recognition boxes that meet the requirements within each cluster according to the positional relationship and the category relationship, the target word order arrangement group of the text recognition boxes is obtained. The step of calculating the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster includes: The average angle within the same cluster is used as the coordinate system transformation angle within that cluster. Map the rotation angle of each text recognition box within the same cluster to a preset coordinate system; Based on the preset coordinate system, the normalized threshold between each pair of text recognition boxes is calculated to determine whether the pairs of text recognition boxes can be concatenated. The rotation angles of the multiple text recognition boxes are then clustered using DBScan to obtain multiple clusters, including: Determine whether the difference in rotation angle between any two text recognition boxes is greater than a preset threshold; If it is greater than, then do not connect; If the value is less than 1, the two text recognition boxes are connected to obtain multiple clusters, and each cluster includes at least one text recognition box.

2. The method according to claim 1, characterized in that, The calculation of the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text, and previously also includes: Obtain images containing at least the text information of the marketing poster packaging; Based on OCR, semantically coherent text in the text information image is labeled as a text recognition box, and the coordinate information of the four corner points of each text recognition box is determined.

3. The method according to claim 2, characterized in that, The calculation of the rotation angle of each text recognition box includes: Based on the coordinate information of the four corner points of each text recognition box, calculate the vector corresponding to the preset corner point; The angle between the vector and the unit vector of the preset coordinate axis is obtained by the cross product between the different vectors.

4. A text recognition device, characterized in that, The device includes: A calculation module is used to calculate the rotation angle of each text recognition box, wherein the text recognition box includes at least a set of semantically coherent text; The clustering module is used to perform DBScan clustering on the rotation angles of multiple text recognition boxes to obtain multiple clusters; The arrangement module is used to separate the non-compliant text recognition boxes in each cluster and arrange the compliant text recognition boxes in each cluster according to a preset word order to obtain a target word order arrangement group. The matching module is used to match the target word order arrangement group with product tags in a preset dictionary to obtain the recognition result; the separation of non-compliant text recognition boxes within each cluster includes: Calculate the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster; Based on the normalization threshold, after separating the text recognition boxes that do not meet the requirements, the remaining text within the clusters is reclassified to obtain the minimum clusters. Each cluster includes at least two minimum clusters, and each minimum cluster includes at least one text recognition box. The text recognition boxes that meet the requirements within each cluster are arranged according to a preset word order to obtain a target word order arrangement group, including: Using text reading habits as prior knowledge, DBScan clusters the text recognition boxes that meet the requirements within each cluster in a preset coordinate system; Based on the DBScan clustering results, determine the positional and categorical relationships between the text recognition boxes that meet the requirements within each cluster. The positional relationships include vertical and horizontal relationships, and the categorical relationships include distance relationships. After arranging the text recognition boxes that meet the requirements within each cluster according to the positional relationship and the category relationship, the target word order arrangement group of the text recognition boxes is obtained. The step of calculating the normalization threshold between each pair of text recognition boxes based on each text recognition box within the cluster includes: The average angle within the same cluster is used as the coordinate system transformation angle within that cluster. Map the rotation angle of each text recognition box within the same cluster to a preset coordinate system; Based on the preset coordinate system, the normalized threshold between each pair of text recognition boxes is calculated to determine whether the pairs of text recognition boxes can be concatenated. The rotation angles of the multiple text recognition boxes are then clustered using DBScan to obtain multiple clusters, including: Determine whether the difference in rotation angle between any two text recognition boxes is greater than a preset threshold; If it is greater than, then do not connect; If the value is less than 1, the two text recognition boxes are connected to obtain multiple clusters, and each cluster includes at least one text recognition box.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 3 when executed.

6. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 3.

Images