A text typesetting method, apparatus, storage medium, and computer device

Abstract

This application discloses a text typesetting method, apparatus, storage medium, and computer device. This application can acquire multiple text boxes in a target text image, and the text content of each text box; determine the geometric information of each text box; based on the geometric information of each text box, sort all text boxes in the target text image according to a target arrangement direction to obtain the order information of each text box; calculate the association information between adjacent text boxes in the target text image; based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, determine the typesetting information of the text content of each text box; and typeset all text content of the target text image according to the typesetting information of the text content of each text box. This application can typeset the text content of a target text image.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a text typesetting method, apparatus, storage medium, and computer equipment. Background Technology

[0002] With the development of technology, related technologies in the field of computer vision can be used to recognize images containing text and obtain recognition results. In existing technologies, the recognition results can include editable text, but they cannot effectively recognize the layout information of the editable text in the image, resulting in a large difference between the editable text and its layout in the image.

[0003] In the process of researching and practicing existing technologies, the inventors of this application discovered that due to the significant difference between editable text and its layout in images, users need to manually format the editable text before using it. Summary of the Invention

[0004] This application provides a text typesetting method, apparatus, storage medium, and computer device that can typeset the text content in a target text image.

[0005] This application provides a text typesetting method, including:

[0006] Retrieve multiple text boxes from a target text image, and the text content of each text box;

[0007] Determine the geometric information for each text box;

[0008] Based on the geometric information of each text box, all text boxes in the target text image are sorted according to the target arrangement direction to obtain the order information of each text box;

[0009] Calculate the association information between adjacent text boxes in the target text image;

[0010] Based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, the layout information of the text content of each text box is determined;

[0011] Based on the layout information of the text content in each text box, all text content of the target text image is formatted.

[0012] Accordingly, this application provides a text typesetting apparatus, comprising:

[0013] The acquisition module is used to acquire multiple text boxes in the target text image, as well as the text content of each text box;

[0014] The geometry determination module is used to determine the geometric information of each text box;

[0015] The sorting module is used to sort all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box, so as to obtain the order information of each text box;

[0016] The calculation module is used to calculate the association information between adjacent text boxes in the target text image;

[0017] The information determination module is used to determine the layout information of the text content of each text box based on the geometric information, order information and association information between all adjacent text boxes in the target text image;

[0018] The layout module is used to layout all the text content of the target text image according to the layout information of the text content of each text box.

[0019] In some embodiments, the geometric information includes border information and feature information, and the geometry determination module includes a measurement submodule and a determination submodule, wherein,

[0020] The measurement submodule is used to measure the text box and obtain the border information of the text box;

[0021] The determination submodule is used to determine the feature information of the text box based on the border information.

[0022] In some embodiments, the feature information includes feature lines, feature points, and the angle values ​​of the feature lines. The determining submodule is specifically used for:

[0023] Based on the border information, the feature points and feature lines of the text box are determined within the area where the text box is located;

[0024] Measure the angle value of the feature line in the reference direction of the target text image.

[0025] In some embodiments, the target arrangement direction includes a first target direction and a second target direction, and the sorting module includes a determining submodule, a sorting submodule, and a setting submodule, wherein,

[0026] The determination submodule is used to determine the starting text box and the starting feature point of the starting text box from all text boxes in the target text image;

[0027] The sorting submodule is used to sort the feature points of all text boxes in the first target direction and the second target direction of the target text image, starting from the initial feature point, to obtain the sorting information of the feature points of each text box.

[0028] The setting submodule is used to set the sorting of feature points to the order information of the text boxes to which the feature points belong, so as to obtain the order information of each text box.

[0029] In some embodiments, the sorting submodule is specifically used for:

[0030] Starting from the initial feature point, at least one target sub-direction of the target first direction in the target image is determined in the target image along the second target direction;

[0031] Starting from the initial feature point, all feature points along the target sub-direction are sorted to obtain the sorting information of all feature points along the target sub-direction.

[0032] In some embodiments, the adjacent text boxes include a first text box and a second text box, the association information includes distance information and ratio information, and the calculation module includes a calculation submodule and a determination submodule, wherein...

[0033] The calculation submodule is used to calculate the distance information between the first text box and the second text box based on the feature lines of the first text box and the feature points of the second text box.

[0034] The determination submodule is used to determine the ratio information between the first text box and the second text box based on the feature lines of the first text box and the second text box.

[0035] In some embodiments, determining the submodule is specifically used for:

[0036] Determine the first projection length of the feature line of the first text box in the reference direction;

[0037] Determine the second projection length of the feature line of the second text box in the reference direction;

[0038] Calculate the length ratio information between the first projection length and the second projection length, and set the length ratio information as the ratio information between the first text box and the second text box.

[0039] In some embodiments, the association information includes distance information and proportion information, the adjacent text boxes include a first text box and a second text box, and the information determination module is specifically used for:

[0040] Calculate the angle difference between the angle values ​​of the feature lines of the first text box and the second text box;

[0041] When the angle difference is less than a preset angle threshold, the distance information is less than a preset distance threshold, and the ratio information is greater than a preset ratio threshold, the same layout information is assigned to the first text box and the second text box.

[0042] When the distance information is greater than a preset distance threshold, different layout information is assigned to the first text box and the second text box.

[0043] In some embodiments, the sorting module is specifically used for:

[0044] The target image is segmented to obtain at least two content blocks, each content block including a text block, and each text block including at least one text box.

[0045] Based on the geometric information of each text box, the at least one text box in the text block is sorted according to the target arrangement direction to obtain the order information of each text box in the text block;

[0046] The calculation module is specifically used for:

[0047] Calculate the association information between adjacent text boxes within at least one text block of the target text image;

[0048] In some embodiments, the content block includes a non-text block, the non-text block corresponding to non-text content, and the information determination module is specifically used for:

[0049] Based on the geometric information, order information, and association information between all text boxes within the text block of the target text image, the layout information of the text content of each text box within the text block is determined.

[0050] The typesetting module is specifically used for:

[0051] The layout information of the text content in each text box within the text block, as well as the position information of the non-text blocks, are used to format all text content and non-text content within the target text image.

[0052] Accordingly, this application also provides a storage medium storing a computer program adapted for loading by a processor to execute any of the text typesetting methods provided in this application.

[0053] Accordingly, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the text typesetting methods provided in this application.

[0054] This application can acquire multiple text boxes in a target text image, as well as the text content of each text box; determine the geometric information of each text box; based on the geometric information of each text box, sort all text boxes in the target text image according to the target arrangement direction to obtain the order information of each text box; calculate the association information between adjacent text boxes in the target text image; based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, determine the layout information of the text content of each text box; and layout all text content in the target text image according to the layout information of the text content of each text box.

[0055] After receiving multiple text boxes of a target sample image and the text content of each text box, this application can perform information analysis and sorting on the text boxes, and determine the layout information of the text content of each text box based on the obtained geometric information, sequence information and correlation information, thereby realizing the layout of the text content of the target text image. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of a scenario for the text typesetting system provided in an embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating the text typesetting method provided in an embodiment of this application;

[0059] Figure 3 This is a process example diagram of the text typesetting method provided in the embodiments of this application;

[0060] Figure 4 This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0061] Figure 5 This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0062] Figure 6 This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0063] Figure 7 This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0064] Figure 8This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0065] Figure 9 This is another process example diagram of the text typesetting method provided in the embodiments of this application;

[0066] Figure 10 This is another flowchart illustrating the text typesetting method provided in the embodiments of this application;

[0067] Figure 11 This is a schematic diagram of the structure of the text typesetting device provided in the embodiments of this application;

[0068] Figure 12 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described in this application are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0071] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0072] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0073] In this application, the text boxes and text content can be obtained by recognizing the target text image using computer vision technology.

[0074] The text typesetting method of this application can be integrated into a text typesetting device, which can be integrated into a text typesetting system. The text typesetting system can include one or more computer devices, which can include terminals or servers, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0075] See Figure 1 The text typesetting system may include a text typesetting device, which may be integrated into a computer device. The text typesetting device may acquire multiple text boxes in a target text image, as well as the text content of each text box; determine the geometric information of each text box; based on the geometric information of each text box, sort all text boxes in the target text image according to the target arrangement direction to obtain the order information of each text box; calculate the association information between adjacent text boxes in the target text image; based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, determine the typesetting information of the text content of each text box; and typeset all the text content of the target text image according to the typesetting information of the text content of each text box.

[0076] It should be noted that, Figure 1The schematic diagram of the text typesetting system shown is merely an example. The text typesetting system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of text typesetting devices and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0077] The following sections provide detailed explanations. In this embodiment, a text formatting method will be described in detail. This text formatting method can be integrated into a server, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating the text formatting method provided in an embodiment of this application. The text formatting method may include:

[0078] 101. Obtain multiple text boxes in the target text image, and the text content of each text box.

[0079] The target text image can include images containing text. The target text image can include images captured on a computer device, images taken with a camera or other device, images received from others, etc. The target text image can include various formats, such as JPEG, BMP, GIF, PNG, etc.

[0080] Image recognition of a target text image can yield recognition results, which may include multiple text boxes in the target text image and the text content within each text box. Specifically, the recognition of the target text image can be based on Optical Character Recognition (OCR) technology.

[0081] The text box may include a region containing text in the target text image, and the text content may include the editable text content within the text box. The text in this application may include text in multiple languages, including English and Chinese, multiple data texts, including Arabic numerals and Roman numerals, and multiple symbolic texts, including punctuation marks and mathematical symbols. The text in this application may include handwritten text, printed text, etc.

[0082] Specifically, there are various ways to obtain the text box and its text content, such as obtaining it directly from the local computer device, or sending a request to another computer device (such as a server) and receiving a data packet containing the text box and its text content from the other computer device, and so on.

[0083] For example, see Figure 3The process involves acquiring six text boxes from image 1 and the text content of each text box. Specifically, image 1 may include text box 1, text box 2, text box 3, text box 4, text box 5, and text box 6. The text content of text box 1 is "1.2.2.XXXX II of XXXX", the text content of text box 2 is "Based on deep learning, using XXXX to identify text segment regions, table regions, image regions, etc. XXXX comparison", the text content of text box 3 is "Difficult to obtain and dependent on training data, for XXXXXXXXXXXXXX.", the text content of text box 4 is "2. Detailed explanation of XXXXXXX (XXXX)", the text content of text box 5 is "2.1. Complete XXXX provided by XXX (XXXX)", and the text content of text box 6 is "We perform XXXX based on the results of text recognition. For text lines XXXXXXXXXX, determine paragraphs and spaces."

[0084] 102. Determine the geometric information of each text box.

[0085] The geometric information can include the graphic information of the text box. If the text box can be a rectangle, then the geometric information can include the position information of the points in the text box, the position information of the lines, the length information, etc.

[0086] Specifically, there are several ways to determine the geometric information of a text box, such as retrieving it directly from memory or measuring the text box, etc.

[0087] For example, targeting Figure 3 The text box 1 in the text box can be measured and calculated to obtain the geometric information 1 of the text box 1.

[0088] In some embodiments, the geometric information includes border information and feature information, and the step "determine the geometric information of each text box" may include:

[0089] Measure the text box to obtain its border information;

[0090] Based on the border information, the feature information of the text box is determined.

[0091] The border information may include the position and length of the text box's edges, and the feature information may include information that reflects and represents some or all of the text box's features.

[0092] There are several ways to determine the feature information of a text box based on its border information. For example, specific border information can be set as the feature information of the text box, specific border information can be obtained by filtering all border information of the text box based on preset conditions, specific border information can also be randomly selected from all border information, and so on.

[0093] For example, the text box 1 can be measured to obtain the border information 1 of the text box 1. The border information 1 can include the position and length of each side of the text box 1 in the image 1. Then, the feature information 1 of the text box can be determined based on the border information 1.

[0094] In some embodiments, the feature information includes feature lines, feature points, and the angle values ​​of the feature lines. The step "determine the feature information of the text box based on the border information" may include:

[0095] Based on the border information, the feature points and feature lines of the text box are determined within the area where the text box is located;

[0096] Measure the angle value of the feature line in the reference direction of the target text image.

[0097] Among them, feature points include points within the text box. Feature points can represent some features of the text box, such as the center point, center of gravity, and center of the text box.

[0098] Among them, feature lines can include lines within the text box. Feature lines can represent some features of the text box. For example, feature lines can include line segments connecting any two points within the text box.

[0099] Specifically, the text box can be a triangle, quadrilateral, polygon, or circle, etc. When the text box is a rectangle, the border information can include the left edge, right edge, top edge, and bottom edge. The method to determine the feature points and feature lines of the text box can be as follows: take the left midpoint and right midpoint of the left edge and right edge of the rectangle respectively, connect the left midpoint and right midpoint to obtain the feature line of the rectangle, determine the midpoint of the feature line, and set the midpoint of the feature line as the feature point of the text box.

[0100] The reference direction can be any direction, and it can be determined according to the direction of the text box in the target sample data. Usually, the reference direction can be a direction that is close to or the same as the direction of the text in the text box. The reference direction can be displayed in the form of a line.

[0101] The angle value can be the angle between the feature line and the reference direction. The angle value can be measured directly or calculated using trigonometric functions, etc. The specific choice can be made flexibly in practical applications, and no restrictions are imposed here.

[0102] For example, see Figure 4The left and right midpoints of the left and right lines of text box 1 can be determined by measurement. Connecting the left and right midpoints will give us the feature line 1 of text box 1. The midpoint of feature line 1 will give us the feature point 1 of text box 1. The reference direction can be the horizontal direction of image 1, i.e., direction 1. The angle between direction 1 and feature line 1 can be measured or calculated to get the angle value 1.

[0103] 103. Based on the geometric information of each text box, sort all text boxes in the target text image according to the target arrangement direction to obtain the order information of each text box.

[0104] The target arrangement direction can include the direction on which the text boxes are sorted. The target arrangement direction can include multiple degrees. For example, the target arrangement direction can be the horizontal direction, the horizontal direction on the coordinate axis with the vertical axis at 20, the direction indicated by the point with the vertical coordinate at 20 and the horizontal coordinate at 0 or greater, and so on. When there are multiple target arrangement directions, each target arrangement direction can include a priority, and the order information of the text boxes is determined according to the priority.

[0105] The sequence information can include the order of the text box among all the text boxes, and the sequence information can be marked in the form of text, numbers, symbols, etc.

[0106] Specifically, the geometric information of text boxes can be detected in the target sorting direction in a sequential order. Smaller order information is set for the text boxes to which the first detected geometric information belongs, and larger order information is set for the text boxes to which the later detected geometric information belongs. For example, if the geometric information can include the center point of the text box, then in the target sorting direction, the order information of the text box to which the first detected center point X belongs is determined as s1.

[0107] For example, see Figure 5 The target sorting direction can be direction 2 (the vertical direction in image 2), and the geometric information on which the sorting is based can be the left frame line of the text box. Then, the left frame line can be detected sequentially in direction 2 to obtain the sorting information of text box 1 (sequence 1), text box 2 (sequence 2), text box 3 (sequence 3), text box 4 (sequence 4), text box 5 (sequence 5), and text box 6 (sequence 6).

[0108] In some embodiments, the target arrangement direction includes a first target direction and a second target direction. The step "based on the geometric information of each text box, sort all text boxes in the target text image according to the target arrangement direction to obtain the order information of each text box" is described.

[0109] From all text boxes in the target text image, determine the starting text box and the starting feature point of the starting text box;

[0110] Starting from the initial feature point, sort the feature points of all text boxes in the first and second target directions of the target text image to obtain the sorting information of the feature points of each text box.

[0111] Set the sorting information of the feature points to the order information of the text boxes to which the feature points belong, so as to obtain the order information of each text box.

[0112] The starting text box and the starting feature point can be the starting point in the sorting process, and the order information of the starting text box can be sequence 1.

[0113] Specifically, if the geometric information can be the feature points of the text box, then the feature points can be detected in the first and second directions of the target, starting from the initial feature point, so as to obtain the sequence information of the feature points of each text box. The sequence information of the feature points is the feature information of the text box to which the feature point belongs.

[0114] For example, the starting text box in image 1 can be text box 1, the starting feature point can be feature point 1 of text box 1, and the target text direction can include direction 1 and direction 2 (corresponding to the first target direction and the second target direction). Then, the feature points of all text boxes can be sorted on direction 1 and direction 2 to obtain the sorting information of each feature point. The feature information of the sorted point is the order information of the text box to which the sorted point belongs.

[0115] In some embodiments, the step "starting from the initial feature point, sorting the feature points of all text boxes along the first and second target directions of the target text image to obtain the position information of the feature points of each text box" may include:

[0116] Starting from the initial feature point in the second direction of the target, at least one sub-direction of the target in the target image is determined in the first direction of the target.

[0117] Starting from the initial feature point, sort all feature points along the target sub-direction to obtain the position information of all feature points along the target sub-direction.

[0118] For example, see Figure 6In direction 2, six sub-directions in direction 1 can be determined, including sub-direction 1, sub-direction 2, sub-direction 3, sub-direction 4, sub-direction 5 and sub-direction 6 from top to bottom in direction 2. Then, starting from feature point 1, the presence of feature points is detected in sub-direction 1, sub-direction 2, sub-direction 3, sub-direction 4, sub-direction 5 and sub-direction 6 in sequence. Based on the order in which feature points are detected, each detected feature point is sorted to obtain the sorting information of each feature point.

[0119] 104. Calculate the association information between adjacent text boxes in the target text image.

[0120] The adjacent text boxes can include text boxes that are adjacent to each other in the target text image. If there are no other text boxes between two text boxes, then these two text boxes are a group of adjacent text boxes. Adjacency can include adjacency in a specific direction. For example, the direction referred to in determining the adjacency relationship can be the target arrangement direction when sorting text boxes.

[0121] The association information may include information on the association between adjacent text boxes in the target text image. The association relationship can be determined by pre-setting or by determining the relevant information of adjacent text boxes. For example, the association relationship between adjacent text boxes can be determined by determining the geometric information of adjacent text boxes. Specifically, at least one benchmark comparison value can be set for the geometric information, and the association relationship can be calculated based on the comparison result between the geometric information and the benchmark comparison value, and so on.

[0122] For example, we can calculate the association information 1 between text box 1 and text box 2, the association information 2 between text box 2 and text box 3, the association information 3 between text box 3 and text box 4, the association information 4 between text box 4 and text box 5, and the association information 5 between text box 5 and text box 6 in image 1.

[0123] In some embodiments, adjacent text boxes include a first text box and a second text box, and the association information includes distance information and scale information. The step of "calculating the association information between adjacent text boxes in the target text image" may include:

[0124] Calculate the distance information between the first text box and the second text box based on the feature lines of the first text box and the feature points of the second text box.

[0125] Based on the feature lines of the first and second text boxes, the proportional information between the first and second text boxes is determined.

[0126] The distance information can include information that can characterize the distance between adjacent text boxes. The distance information can be calculated in various ways, such as by calculating it based on the geometric information of the text boxes (including border information, feature information, etc.), for example, by determining the distance information based on the left edge of the adjacent text boxes, or by calculating it based on the feature points and feature lines of the adjacent text boxes.

[0127] Specifically, any adjacent text boxes can include a first text box and a second text box. The order information of the first text box precedes the order information of the second text box. The feature lines of the first text box and the feature points of the second text box can be determined, and the distance between the feature points and the feature lines can be calculated. The distance obtained is the distance information between the first text box and the second text box.

[0128] The proportional information can include the proportional relationship between the first text box and the second text box. The proportional information can be obtained from the geometric information of the text boxes and can reflect the proportional relationship of certain aspects of the text boxes (such as length, area, etc.).

[0129] For example, the area ratio between the first and second text boxes can be determined based on their border information. Or, the length ratio between the first and second text boxes can be determined based on their feature lines, border information, etc.

[0130] For example, based on the feature line 1 of text box 1 and the feature point 2 of text box 2, calculate the distance information 1 between text box 1 and text box 2, and determine the ratio information 1 between text box 1 and text box 2 based on the feature line 1 of text box 1 and the feature line 2 of text box 2.

[0131] In some embodiments, the step "determining the ratio information between the first text box and the second text box based on the feature lines of the first text box and the second text box" may include:

[0132] Determine the first projection length of the feature line of the first text box in the reference direction;

[0133] Determine the second projection length of the feature line of the second text box in the reference direction;

[0134] Calculate the length ratio information between the first projection length and the second projection length, and set the length ratio information as the ratio information between the first text box and the second text box.

[0135] For example, see Figure 7The reference direction can be direction 1. The projection of feature line 1 of text box 1 onto the reference direction is projection line segment 1, and the length of projection line segment 1 is projection length 1. The projection of feature line 2 of text box 2 onto the reference direction is projection line segment 2, and the length of projection line segment 1 is projection length 2. Calculate the length ratio information between projection length 1 and projection length 2, and set this length ratio information as the ratio information between text box 1 and text box 2.

[0136] 105. Based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, determine the layout information of the text content of each text box.

[0137] The formatting information can include paragraph information of the text content. The paragraph information can include the paragraph in which the text content is located, as well as the spacing (before, after, and within the paragraph), margins (indentation value), and other information of the paragraph. It is possible to first determine the paragraph to which the text content of the text box belongs, and then determine the paragraph information of the text content within that paragraph. When determining the paragraph to which the text content belongs, it can be based on the association information between adjacent text boxes. For example, when the association information meets the preset conditions, it can be determined that the two adjacent text boxes belong to the same paragraph.

[0138] Specifically, when determining the paragraph to which each text box belongs, a starting text box can be determined first, and a paragraph identifier 1 can be assigned to the text content of the starting text box. Then, the paragraph information of the starting text box and its adjacent text boxes can be compared to see if they meet the preset conditions. If they do, the text box is assigned a paragraph identifier 1, which is the same as the paragraph identifier of the starting text box. If they do not meet the conditions, the text box is assigned a paragraph identifier 2, which is a new paragraph identifier different from the starting text box. According to the order information of each text box, the association information of all adjacent text boxes is compared in turn to obtain the paragraph identifier of all text boxes.

[0139] For example, paragraph identifier d1 is assigned to text box 1. Based on the order information of the text boxes, the association information and geometric information of text box 1 and text box 2 in image 1 are compared. If the comparison finds that the preset conditions are not met, paragraph identifier d2 is assigned to text box 2. Then the association information and geometric information of text box 2 and text box 3 are compared. If the preset conditions are met, paragraph identifier d2 is assigned to text box 3, which is the same as that of text box 2. This process continues until the association information and geometric information of all text boxes are compared, resulting in paragraph identifier d3 for text box 4, paragraph identifier d4 for text box 5, and paragraph identifier d5 for text box 6.

[0140] In some embodiments, the association information includes distance information and scale information, and the adjacent text boxes include a first text box and a second text box. The step of "determining the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image" includes:

[0141] Calculate the angle difference between the angle values ​​of the feature lines of the first and second text boxes;

[0142] When the angle difference is less than the preset angle threshold, the distance information is less than the preset distance threshold, and the ratio information is greater than the preset ratio threshold, the same layout information is assigned to the first text box and the second text box.

[0143] When the distance information is greater than the preset distance threshold, different layout information is assigned to the first text box and the second text box.

[0144] Specifically, geometric information includes the angle values ​​of the feature lines of the text boxes. The difference in angles between adjacent text boxes can be calculated. For the text content of the same paragraph, even if they are on different lines, there should be a relatively fixed direction of writing. The angle difference is usually kept within a reasonable range. The distance between different lines in the same paragraph remains relatively stable. Except for the special case that may exist in the last line of the paragraph, the difference in the amount of text in different lines also remains relatively stable. Even if operations such as screenshotting or taking pictures may cause a certain degree of distortion in the target text image, it is still within a reasonable range. Therefore, by comparing angle differences, distance information, and scale information, it can be determined whether adjacent text boxes are in the same paragraph.

[0145] Among them, the preset angle threshold, preset distance threshold, and preset ratio threshold can all be flexibly set.

[0146] If the distance between adjacent text boxes is large, indicating that the text content of these two text boxes belongs to different paragraphs or there is a blank line in between, it is necessary to segment or fill in blank lines based on the distance information.

[0147] For example, in image 2, if the distance between text box 1 and text box 2 is greater than a preset distance threshold 2, then text box 2 is assigned a different paragraph identifier A than text box 1. If the angle difference between text box 2 and text box 3 is less than a preset angle threshold 1, the distance between text box 2 and text box 3 is less than a preset distance threshold 2, and the ratio between text box 2 and text box 3 is greater than a preset ratio threshold 3, then text box 3 can be assigned the same paragraph identifier A as text box 2.

[0148] 106. Based on the layout information of the text content in each text box, format all the text content of the target text image.

[0149] For example, based on the layout information of the text content in each text box in Image 1, Image 1 is re-formatted, and the resulting content can be found in [reference needed]. Figure 9 .

[0150] In some embodiments, the step "based on the geometric information of each text box, sorting all text boxes in the target text image according to the target arrangement direction to obtain the order information of each text box" may include:

[0151] The target image is segmented to obtain at least two content blocks, each content block including a text block, and each text block including at least one text box.

[0152] Based on the geometric information of each text box, at least one text box is sorted in the text block according to the target arrangement direction to obtain the order information of each text box in the text block;

[0153] The step "Calculate the association information between adjacent text boxes in the target text image" may include:

[0154] Calculate the association information between adjacent text boxes within at least one text block of the target text image.

[0155] In some embodiments, the content block includes a non-text block, which corresponds to non-text content. The step of "determining the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image" may include:

[0156] Based on the geometric information, order information, and association information between all text boxes within the text block of the target text image, determine the layout information of the text content of each text box within the text block.

[0157] The step "Format all text content of the target text image according to the formatting information of the text content in each text box" may include:

[0158] The layout of all text and non-text content within the target text image is determined by using the layout information of the text content in each text box within the text block, as well as the position information of non-text blocks.

[0159] For example, see Figure 8Image 2 is segmented to obtain text block 1, image block 1, and text block 2. Based on the geometric information of each text box in text block 1, all text boxes in text block 1 are sorted according to the target arrangement direction to obtain the order information of each text box in text block 1. The association information between all adjacent text boxes in text block 1 is calculated, including distance information and scale information, and then the layout information of all text content in text block 1 is obtained. Finally, based on the position information of the segmented image block 1 and text block 2, image 2 is laid out.

[0160] After receiving multiple text boxes of a target sample image and the text content of each text box, this application can perform information analysis and sorting on the text boxes, and determine the layout information of the text content of each text box based on the obtained geometric information, sequence information and correlation information, thereby realizing the layout of the text content of the target text image.

[0161] Based on the method described in the above embodiments, the following examples will provide further detailed explanations.

[0162] This application will use a text typesetting system integrated into a computer device as an example to introduce text typesetting methods, such as... Figure 10 As shown, Figure 10 This is a flowchart illustrating the text formatting method provided in an embodiment of this application. The text formatting method may include:

[0163] 201. The computer equipment recognizes the target image and obtains multiple text boxes and the text content of each text box.

[0164] 202. The computer equipment determines the feature lines and feature points of each text box.

[0165] 203. The computer equipment measures the angle value of the feature line of each text box in the reference direction.

[0166] 204. A computer device segments an image to obtain at least one text block, and the text block includes at least one text box.

[0167] 205. The computer equipment sorts the feature points in all text blocks in the target image according to the target arrangement direction, obtains the sorting information of each feature point in each text block, and determines the order information of the text box to which each feature point belongs in the text block.

[0168] 206. The computer device calculates the distance and proportion information between all adjacent text boxes within each text block.

[0169] 207. The computer device determines the layout information of the text content of each text box in each text block based on the feature lines, feature points, angle values ​​and order information of each text box in each text block, as well as the distance information and proportion information between all adjacent text boxes.

[0170] 208. The computer equipment typesets all the text content in the target image according to the layout information of the text content in each text box in each text block.

[0171] After receiving multiple text boxes of a target sample image and the text content of each text box, this application can perform information analysis and sorting on the text boxes, and determine the layout information of the text content of each text box based on the obtained geometric information, sequence information and correlation information, thereby realizing the layout of the text content of the target text image.

[0172] To facilitate better implementation of the text typesetting method provided in this application, this application also provides an apparatus based on the above-described text typesetting method. The meanings of the terms used are the same as in the text typesetting method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0173] like Figure 11 As shown, Figure 11 A schematic diagram of the structure of a text typesetting device provided in one embodiment of this application is shown. The text typesetting device may include an acquisition module 301, a geometry determination module 302, a sorting module 303, a calculation module 304, an information determination module 305, and a typesetting module 306.

[0174] The acquisition module 301 is used to acquire multiple text boxes in the target text image, as well as the text content of each text box;

[0175] The geometry determination module 302 is used to determine the geometric information of each text box;

[0176] The sorting module 303 is used to sort all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box, so as to obtain the order information of each text box.

[0177] Calculation module 304 is used to calculate the association information between adjacent text boxes in the target text image;

[0178] The information determination module 305 is used to determine the layout information of the text content of each text box based on the geometric information, order information and association information between all adjacent text boxes in the target text image;

[0179] The layout module 306 is used to layout all the text content of the target text image according to the layout information of the text content of each text box.

[0180] In some embodiments, the geometric information includes boundary information and feature information, and the geometry determination module includes a measurement submodule and a determination submodule, wherein...

[0181] The measurement submodule is used to measure the text box and obtain the border information of the text box;

[0182] The determination submodule is used to determine the feature information of the text box based on the border information.

[0183] In some embodiments, the feature information includes feature lines, feature points, and the angle values ​​of the feature lines. The determining submodule is specifically used for:

[0184] Based on the border information, the feature points and feature lines of the text box are determined within the area where the text box is located;

[0185] Measure the angle value of the feature line in the reference direction of the target text image.

[0186] In some embodiments, the target arrangement direction includes a first target direction and a second target direction, and the sorting module includes a determining submodule, a sorting submodule, and a setting submodule, wherein...

[0187] The determination submodule is used to determine the starting text box and the starting feature point of the starting text box from all text boxes in the target text image;

[0188] The sorting submodule is used to sort the feature points of all text boxes in the first and second target directions of the target text image, starting from the initial feature point, to obtain the sorting information of the feature points of each text box.

[0189] The settings submodule is used to set the sorting of feature points to the order information of the text boxes to which the feature points belong, so as to obtain the order information of each text box.

[0190] In some embodiments, the sorting submodule is specifically used for:

[0191] Starting from the initial feature point in the second direction of the target, at least one sub-direction of the target in the target image is determined in the first direction of the target.

[0192] Starting from the initial feature point, sort all feature points along the target sub-direction to obtain the sorting information of all feature points along the target sub-direction.

[0193] In some embodiments, adjacent text boxes include a first text box and a second text box, and the associated information includes distance information and ratio information. The calculation module includes a calculation submodule and a determination submodule, wherein...

[0194] The calculation submodule is used to calculate the distance information between the first text box and the second text box based on the feature lines of the first text box and the feature points of the second text box;

[0195] The determination submodule is used to determine the ratio information between the first text box and the second text box based on the feature lines of the first text box and the second text box.

[0196] In some embodiments, determining the submodule is specifically used for:

[0197] Determine the first projection length of the feature line of the first text box in the reference direction;

[0198] Determine the second projection length of the feature line of the second text box in the reference direction;

[0199] Calculate the length ratio information between the first projection length and the second projection length, and set the length ratio information as the ratio information between the first text box and the second text box.

[0200] In some embodiments, the associated information includes distance information and ratio information, the adjacent text boxes include a first text box and a second text box, and the information determination module is specifically used for:

[0201] Calculate the angle difference between the angle values ​​of the feature lines of the first and second text boxes;

[0202] When the angle difference is less than the preset angle threshold, the distance information is less than the preset distance threshold, and the ratio information is greater than the preset ratio threshold, the same layout information is assigned to the first text box and the second text box.

[0203] When the distance information is greater than the preset distance threshold, different layout information is assigned to the first text box and the second text box.

[0204] In some embodiments, the sorting module is specifically used for:

[0205] The target image is segmented to obtain at least two content blocks, each content block including a text block, and each text block including at least one text box.

[0206] Based on the geometric information of each text box, at least one text box is sorted in the text block according to the target arrangement direction to obtain the order information of each text box in the text block;

[0207] The calculation module is specifically used for:

[0208] Calculate the association information between adjacent text boxes within at least one text block of the target text image;

[0209] In some embodiments, the content block includes a non-text block, which corresponds to non-text content. The information determination module is specifically used for:

[0210] Based on the geometric information, order information, and association information between all text boxes within the text block of the target text image, determine the layout information of the text content of each text box within the text block.

[0211] The typesetting module is specifically used for:

[0212] The layout of all text and non-text content within the target text image is determined by using the layout information of the text content in each text box within the text block, as well as the position information of non-text blocks.

[0213] In this application, the acquisition module 301 acquires multiple text boxes in the target text image and the text content of each text box; the geometry determination module 302 determines the geometric information of each text box; the sorting module 303 sorts all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box to obtain the order information of each text box; the calculation module 304 calculates the association information between adjacent text boxes in the target text image; the information determination module 305 determines the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image; and the layout module 306 typesets all the text content of the target text image according to the layout information of the text content of each text box.

[0214] After receiving multiple text boxes of a target sample image and the text content of each text box, this application can perform information analysis and sorting on the text boxes, and determine the layout information of the text content of each text box based on the obtained geometric information, sequence information and correlation information, thereby realizing the layout of the text content of the target text image.

[0215] Furthermore, embodiments of this application also provide a computer device, which can be a terminal or a server, such as... Figure 12 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0216] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 12 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0217] The processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0218] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0219] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0220] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0221] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0222] The process involves: acquiring multiple text boxes in a target text image and the text content of each text box; determining the geometric information of each text box; sorting all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box to obtain the order information of each text box; calculating the association information between adjacent text boxes in the target text image; determining the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image; and formatting all text content in the target text image according to the layout information of the text content of each text box.

[0223] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0224] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0225] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0226] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the text typesetting methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0227] The process involves: acquiring multiple text boxes in a target text image and the text content of each text box; determining the geometric information of each text box; sorting all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box to obtain the order information of each text box; calculating the association information between adjacent text boxes in the target text image; determining the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image; and formatting all text content in the target text image according to the layout information of the text content of each text box.

[0228] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0229] Since the computer program stored in the storage medium can execute the steps in any of the text typesetting methods provided in the embodiments of this application, the beneficial effects that any of the text typesetting methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0230] The foregoing has provided a detailed description of a text typesetting method, apparatus, storage medium, and computer device provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A text typesetting method, characterized in that, include: Retrieve multiple text boxes from a target text image, and the text content of each text box; Determine the geometric information of each text box, including: measuring the text box to obtain its border information; determining the feature information of the text box based on the border information; wherein the feature information includes feature lines, feature points, and the angle values ​​of the feature lines; wherein the feature lines are obtained by connecting the left midpoint and the right midpoint of the text box; the midpoint of the feature lines is determined and set as the feature point of the text box. Based on the geometric information of each text box, all text boxes in the target text image are sorted according to the target arrangement direction to obtain the order information of each text box; Calculating the association information between adjacent text boxes in the target text image, wherein the association information includes distance information and ratio information, and the adjacent text boxes include a first text box and a second text box, including: determining the feature line of the first text box and the feature point of the second text box, and calculating the distance between the feature point and the feature line, the resulting distance being the distance information between the first text box and the second text box; determining the first projection length of the feature line of the first text box in the reference direction; determining the second projection length of the feature line of the second text box in the reference direction; calculating the length ratio information between the first projection length and the second projection length, and setting the length ratio information as the ratio information between the first text box and the second text box; Based on the geometric information, order information, and association information between all adjacent text boxes in the target text image, the layout information of the text content of each text box is determined, including: calculating the angle difference between the angle values ​​of the feature lines of the first text box and the second text box; when the angle difference is less than a preset angle threshold, the distance information is less than a preset distance threshold, and the ratio information is greater than a preset ratio threshold, the first text box and the second text box are assigned the same layout information; when the distance information is greater than a preset distance threshold, the first text box and the second text box are assigned different layout information. Based on the layout information of the text content in each text box, all text content of the target text image is formatted.

2. The method according to claim 1, characterized in that, The geometric information includes border information and feature information.

3. The method according to claim 2, characterized in that, The step of determining the feature information of the text box based on the border information includes: Based on the border information, the feature points and feature lines of the text box are determined within the area where the text box is located; Measure the angle value of the feature line in the reference direction of the target text image.

4. The method according to claim 3, characterized in that, The target arrangement direction includes a first target direction and a second target direction. Based on the geometric information of each text box, all text boxes in the target text image are sorted according to the target arrangement direction to obtain the order information of each text box, including: From all text boxes in the target text image, determine the starting text box and the starting feature point of the starting text box; Starting from the initial feature point, the feature points of all text boxes are sorted in the first target direction and the second target direction of the target text image to obtain the sorting information of the feature points of each text box. The sorting of feature points is set as the order information of the text boxes to which the feature points belong, so as to obtain the order information of each text box.

5. The method according to claim 1, characterized in that, Based on the geometric information of each text box, all text boxes in the target text image are sorted according to the target arrangement direction to obtain the order information of each text box, including: The target text image is segmented to obtain at least two content blocks, each content block including a text block, and each text block including at least one text box; Based on the geometric information of each text box, the at least one text box in the text block is sorted according to the target arrangement direction to obtain the order information of each text box in the text block; The calculation of the association information between adjacent text boxes in the target text image includes: Calculate the association information between adjacent text boxes within at least one text block of the target text image.

6. The method according to claim 5, characterized in that, The content block includes non-text blocks, each corresponding to non-text content. The step of determining the layout information of the text content for each text box based on the geometric and sequence information of each text box in the target text image, as well as the association information between all adjacent text boxes, includes: Based on the geometric information, order information, and association information between all text boxes within the text block of the target text image, the layout information of the text content of each text box within the text block is determined. The step of formatting all text content of the target text image according to the formatting information of the text content of each text box includes: The layout information of the text content in each text box within the text block, as well as the position information of the non-text blocks, are used to format all text content and non-text content within the target text image.

7. The method according to claim 1, characterized in that, Based on the geometric information of each text box, all text boxes in the target text image are sorted according to the target arrangement direction to obtain the order information of each text box, including: determining at least one target sub-direction of the target first direction in the target text image with the starting feature point as the starting point in the second target direction; and sorting all feature points in the target sub-direction with the starting feature point as the starting point to obtain the order information of all feature points in the target sub-direction.

8. A text typesetting device, characterized in that, include: The acquisition module is used to acquire multiple text boxes in the target text image, as well as the text content of each text box; The geometry determination module is used to determine the geometric information of each text box, including: measuring the text box to obtain its border information; determining the feature information of the text box based on the border information; wherein the feature information includes feature lines, feature points, and the angle values ​​of the feature lines; wherein the feature lines are obtained by connecting the left midpoint and the right midpoint of the text box; determining the midpoint of the feature lines and setting the midpoint of the feature lines as the feature points of the text box. The sorting module is used to sort all text boxes in the target text image according to the target arrangement direction based on the geometric information of each text box, so as to obtain the order information of each text box; A calculation module is used to calculate the association information between adjacent text boxes in the target text image; wherein the association information includes distance information and ratio information, and the adjacent text boxes include a first text box and a second text box, comprising: determining the feature line of the first text box and the feature point of the second text box, and calculating the distance between the feature point and the feature line, the obtained distance being the distance information between the first text box and the second text box; determining the first projection length of the feature line of the first text box in a reference direction; determining the second projection length of the feature line of the second text box in the reference direction; calculating the length ratio information between the first projection length and the second projection length, and setting the length ratio information as the ratio information between the first text box and the second text box; The information determination module is used to determine the layout information of the text content of each text box based on the geometric information, order information, and association information between all adjacent text boxes in the target text image. This includes: calculating the angle difference between the angle values ​​of the feature lines of the first text box and the second text box; assigning the same layout information to the first text box and the second text box when the angle difference is less than a preset angle threshold, the distance information is less than a preset distance threshold, and the ratio information is greater than a preset ratio threshold; and assigning different layout information to the first text box and the second text box when the distance information is greater than a preset distance threshold. The layout module is used to layout all the text content of the target text image according to the layout information of the text content of each text box.

9. A storage medium, characterized in that, The storage medium stores multiple computer programs adapted for loading by a processor to perform the steps of the method as claimed in any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method as claimed in any one of claims 1 to 7.

Images