Tilt correction method and device for borderless table and storage medium
By using text detection and perspective transformation correction methods, the outer frame is generated by utilizing the tilt angle and vertex coordinates of the rectangle, which solves the problem of inaccurate acquisition of feature points in borderless table images and improves the tilt correction effect.
Patent Information
- Application Number
- CN202310595012.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-11-26
Smart Images

Figure CN119027952A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method, a device and a storage medium for correcting the tilt of a borderless table. Background Art
[0002] When processing documents, it is often necessary to recognize text printed on paper. The usual practice is to take a picture of the paper to obtain an image containing text content, and then use optical character recognition (OCR) technology to recognize the image and obtain the text content. For example, for an image containing a table, it is necessary to perform table structure analysis and text recognition on the image to obtain the cells in the table and the text content in the cells.
[0003] When shooting images containing text content, it is easy to be skewed. The prior art usually uses perspective transformation processing to correct the tilt of the image. For images of tables with complete borders, it is easy to obtain feature points for perspective transformation processing. In practical applications, some tables may be images of borderless tables, that is, the horizontal and vertical lines in the table are omitted. For images of borderless tables, it is difficult to obtain feature points for perspective transformation processing, or the obtained feature points are not accurate enough, which is not conducive to subsequent image tilt correction and table recognition. Summary of the invention
[0004] At least one embodiment of the present application provides a method, device and storage medium for tilt correction of a borderless table, which can improve the accuracy of feature points of the obtained borderless table image and improve the performance of image tilt correction processing.
[0005] In order to solve the above technical problems, this application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a method for correcting the tilt of a borderless table, comprising:
[0007] Obtaining a first rectangular image of a borderless table, wherein the borderless table is filled with text;
[0008] Performing text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment;
[0009] For each edge of the first rectangular image, a rectangular frame set is generated respectively, the rectangular frame set includes at least one rectangular frame adjacent to the edge; according to the inclination angle of the target frame of the rectangular frame in the rectangular frame set, a target inclination angle is determined, the target frame is the frame of the rectangular frame that is most adjacent to the edge; according to the target vertices of each rectangular frame in the rectangular frame set, a reference point is determined, the target vertex is the vertex through which the target frame passes; a straight line passing through the reference point and having an inclination angle of the target inclination angle is generated as an outer frame line of the borderless table;
[0010] According to the four outer frame lines of the borderless table, four intersection points are determined;
[0011] Perform perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix;
[0012] The first rectangular image is corrected according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0013] Optionally, for each edge of the first rectangular image, a rectangular frame set is generated respectively, including:
[0014] Determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold;
[0015] For the left side of the first rectangular image, select the rectangular frames in the first column to form a first rectangular frame set; for the right side of the first rectangular image, select the rectangular frames in the last column to form a second rectangular frame set; for the top side of the first rectangular image, select the rectangular frames in the first row to form a third rectangular frame set; for the bottom side of the first rectangular image, select the rectangular frames in the last row to form a fourth rectangular frame set.
[0016] Optionally, determining a target tilt angle according to the tilt angle of the target border of the rectangular frame in the rectangular frame set includes:
[0017] Calculate the angle between the target border of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line;
[0018] A first statistical value of an angle between a target frame of each rectangular frame and a reference line is calculated as the target tilt angle, wherein the first statistical value is a median or an average value of the angle.
[0019] Optionally, a reference point is determined according to the target vertices of each rectangular box in the rectangular box set, including:
[0020] Get the coordinates of the target vertices of each rectangular box in the rectangular box set;
[0021] A second statistical value of the acquired coordinates of the target vertex is calculated as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
[0022] Optionally, the coordinates of the target vertices of each rectangular box in the rectangular box set are obtained, including:
[0023] For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set;
[0024] For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set;
[0025] For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set;
[0026] With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
[0027] In a second aspect, an embodiment of the present application provides a device for correcting the tilt of a borderless table, comprising:
[0028] An image acquisition module, used to acquire a first rectangular image of a borderless table, wherein the borderless table is filled with text;
[0029] A text detection module, used to perform text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment;
[0030] The outer frame line generation module is used to generate a rectangular frame set for each edge of the first rectangular image, wherein the rectangular frame set includes at least one rectangular frame adjacent to the edge; determine a target tilt angle according to the tilt angle of the target border of the rectangular frame in the rectangular frame set, wherein the target border is the frame line in the rectangular frame that is closest to the edge; determine a reference point according to the target vertices of each rectangular frame in the rectangular frame set, wherein the target vertex is the vertex through which the target border passes; generate a straight line passing through the reference point and having an tilt angle equal to the target tilt angle as an outer frame line of the borderless table;
[0031] An intersection point generation module, used to determine four intersection points according to the four outer frame lines of the borderless table;
[0032] A matrix acquisition module, used for performing perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix;
[0033] The image correction module is used to correct the first rectangular image according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0034] Optionally, the outer frame line generation module includes:
[0035] A first determination module is used to determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold;
[0036] A first selection module, configured to select rectangular frames in a first column from the left side of the first rectangular image to form a first rectangular frame set;
[0037] A second selection module is used to select the last column of rectangular frames on the right side of the first rectangular image to form a second rectangular frame set;
[0038] A third selection module, configured to select rectangular frames in a first row according to the top edge of the first rectangular image to form a third rectangular frame set;
[0039] The fourth selection module is used to select the last row of rectangular frames according to the bottom edge of the first rectangular image to form a fourth rectangular frame set.
[0040] Optionally, the outer frame line generation module further includes:
[0041] A first calculation module, used to calculate the angle between the target frame of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line;
[0042] The second calculation module is used to calculate a first statistical value of the angle between the target border of each rectangular frame and the reference line as the target tilt angle, wherein the first statistical value is the median or average value of the angle.
[0043] Optionally, the outer frame line generation module further includes:
[0044] A first acquisition module is used to acquire the coordinates of the target vertex of each rectangular frame in the rectangular frame set;
[0045] The third calculation module is used to calculate the second statistical value of the acquired coordinates of the target vertex as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
[0046] Optionally, the first acquisition module is further used to:
[0047] For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set;
[0048] For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set;
[0049] For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set;
[0050] With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
[0051] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the steps of the method described above are implemented.
[0052] Compared with the prior art, the tilt correction method and device for borderless tables provided in the embodiments of the present application determine each outer frame line of the borderless table based on the tilt angle and vertex coordinates of the rectangular box obtained by text detection. Since the tilt angles and vertex coordinates of multiple rectangular boxes are comprehensively considered, the embodiments of the present application can improve the accuracy of the feature points of the obtained borderless table image, thereby improving the performance of subsequent image tilt correction processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0054] Figure 1 A flow chart of a method for correcting the tilt of a borderless table according to an embodiment of the present application;
[0055] Figure 2 This is an example diagram of a first rectangular image including a borderless table according to an embodiment of the present application;
[0056] Figure 3An example diagram of a rectangular frame obtained by performing text detection on a first rectangular image;
[0057] Figure 4 This is an example diagram of determining rectangular boxes in the same row in an embodiment of the present application;
[0058] Figure 5 This is an example diagram of determining rectangular frames in the same column in an embodiment of the present application;
[0059] Figure 6 It is a structural schematic diagram of a tilt correction device for a borderless table according to an embodiment of the present application;
[0060] Figure 7 Another structural schematic diagram of the tilt correction device for a borderless table according to an embodiment of the present application. DETAILED DESCRIPTION
[0061] In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.
[0062] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. "And / or" in the specification and claims represents at least one of the connected objects.
[0063] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0064] The following description provides examples and does not limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the functions and arrangements of the elements discussed without departing from the spirit and scope of the present disclosure. Various examples may appropriately omit, replace, or add various procedures or components. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
[0065] In order to improve the accuracy of feature points in the acquired borderless table image, improve the performance of image tilt correction processing, and thus improve the recognition accuracy of the borderless table, the embodiment of the present application provides a tilt correction method for a borderless table, such as Figure 1 As shown, the method includes:
[0066] Step 11, obtaining a first rectangular image of a borderless table, wherein the borderless table is filled with text.
[0067] Here, a borderless table means that there are no horizontal lines and vertical lines in the table. Table 1 provides an example of a borderless table.
[0068] Table 1
[0069]
[0070] In the embodiment of the present application, an image including a borderless table can be obtained by photographing a paper on which a borderless table is printed. Usually, the image obtained by photographing is a rectangular image, and the rectangular image includes four sides, namely a left side, a top side, a right side, and a bottom side. Among them, the left side is opposite to the right side, and the top side is opposite to the bottom side. Figure 2 An example of a first rectangular image captured for a borderless table shown in Table 1 is provided. Figure 2 The first rectangular image 20 shown includes four sides, which are a left side 201 , a top side 202 , a right side 203 and a bottom side 204 . Figure 2 The borderless table in the image is tilted at a certain angle.
[0071] Step 12: Perform text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment.
[0072] Here, the first rectangular image may be subjected to text detection by using the OCR technology of the prior art. The OCR technology can recognize characters in an image, obtain a text segment consisting of at least one character, and generate a rectangular frame circumscribing the text segment. Figure 3 Shows the Figure 2 The detection result obtained by performing text detection on the first rectangular image of the borderless table shown in the figure includes 56 rectangular frames, namely rectangular frames 301 to 356. Since the captured image is skewed to a certain extent, Figure 3 The four borders of the rectangular frame in are not strictly parallel or perpendicular to the sides 201-204 of the first rectangular image. Each rectangular frame has four borders, namely, a left border, an upper border, a right border and a lower border. Each rectangular frame has four vertices, namely, an upper left vertex, an upper right vertex, a lower left vertex and a lower right vertex. The two vertices of the left border refer to the upper left vertex and the lower left vertex. The two vertices of the upper border refer to the upper left vertex and the upper right vertex. The two vertices of the lower border refer to the lower left vertex and the lower right vertex.
[0073] Step 13, for each edge of the first rectangular image, generate a rectangular frame set respectively, the rectangular frame set includes at least one rectangular frame adjacent to the edge; determine a target inclination angle according to the inclination angle of the target border of the rectangular frame in the rectangular frame set, the target border is the frame line of the rectangular frame that is closest to the edge; determine a reference point according to the target vertices of each rectangular frame in the rectangular frame set, the target vertex is the vertex through which the target border passes; generate a straight line passing through the reference point and with an inclination angle of the target inclination angle as an outer frame line of the borderless table.
[0074] Here, for each edge of the first rectangular image, a rectangular frame set is generated, and the rectangular frame set includes at least one rectangular frame adjacent to the edge. In the embodiment of the present application, there are multiple ways to generate the rectangular frame set, which are described in detail below.
[0075] A first manner of generating a rectangular frame set may be to select, for each edge of the first rectangular image, a preset number of rectangular frames closest to the edge to form a rectangular frame set.
[0076] Specifically, first, calculate the distance from the preset vertex of each rectangular box to the edge. For example, for the left side of the first rectangular image, the distance from the upper left vertex or the lower left vertex of the rectangular box to the left side can be calculated; for the top side of the first rectangular image, the distance from the upper left vertex or the upper right vertex of the rectangular box to the top side can be calculated; for the right side of the first rectangular image, the distance from the upper right vertex or the lower right vertex of the rectangular box to the right side can be calculated; for the bottom side of the first rectangular image, the distance from the lower left vertex or the lower right vertex of the rectangular box to the bottom side can be calculated.
[0077] Then, a preset number of rectangular frames are selected in order of distance from small to large to form a set of rectangular frames. The preset number corresponding to each side of the first rectangular image may be the same. For example, for each side of the first rectangular image, three rectangular frames with the smallest distance to the side are selected to form a set of rectangular frames. Figure 3 For example, for the left side 201 of the first rectangular image, three rectangular frames closest to the left side 201, such as rectangular frame 301, rectangular frame 306 and rectangular frame 344, can be selected to form a rectangular frame set. For the top side 202 of the first rectangular image, three rectangular frames closest to the top side 202, such as rectangular frame 301, rectangular frame 302 and rectangular frame 303, can be selected to form a rectangular frame set. For the right side 203 of the first rectangular image, three rectangular frames closest to the right side 203, such as rectangular frame 305, rectangular frame 310 and rectangular frame 314, can be selected to form a rectangular frame set. For the bottom side 204 of the first rectangular image, three rectangular frames closest to the bottom side 204, such as rectangular frames 354, 355 and 356, can be selected to form a rectangular set.
[0078] In the embodiment of the present application, the preset number corresponding to each side of the first rectangular image may also be different or partially the same. For example, for the left side of the first rectangular image, in order of distance from the smallest to the largest, select the four rectangular frames with the smallest distance from the side to form a rectangular frame set; similarly, for the top side of the first rectangular image, select the three rectangular frames with the smallest distance from the side to form a rectangular frame set; for the right side of the first rectangular image, select the four rectangular frames with the smallest distance from the side to form a rectangular frame set; for the bottom side of the first rectangular image, select the three rectangular frames with the smallest distance from the side to form a rectangular frame set.
[0079] A second way to generate a set of rectangular frames may be to select, for each edge of the first rectangular image, a rectangular frame whose distance from the edge does not exceed a preset distance threshold, to form a set of rectangular frames. Specifically, first, calculate the distance from the preset vertex of each rectangular frame to the edge, and the calculation method may refer to the above. Then, for each edge of the first rectangular image, set a corresponding distance threshold. The distance thresholds corresponding to each edge may be the same, partially the same, or completely different. Then, select a rectangular frame whose distance from the edge does not exceed the corresponding distance threshold, to form a set of rectangular frames.
[0080] A third way to generate a set of rectangles is:
[0081] (1) First, determine the rows and columns to which the rectangular frames in the first rectangular image belong, that is, determine the rectangular frames in the same row and the same column in the first rectangular image. Since the image of the borderless table may be skewed, when determining the rows and columns, the row and column relationship can be determined based on the extended line of the border of the rectangular frame.
[0082] Specifically, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than the first threshold, and the distance between the extended lines of the lower borders is less than the second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than the third threshold. The values of the first threshold, the second threshold and the third threshold can be the same or different. Specifically, the values of the thresholds can be set with reference to parameters such as character size, character spacing and line spacing.
[0083] Since the corresponding borders of each rectangular box may not be parallel, the spacing of the upper border extension lines can be expressed by the distance from the two vertices of the upper border of one rectangular box to the upper border extension line of another rectangular box; the spacing of the lower border extension lines can be expressed by the distance from the two vertices of the lower border of one rectangular box to the lower border extension line of another rectangular box; the spacing of the left border extension lines can be expressed by the distance from the two vertices of the left border of one rectangular box to the left border extension line of another rectangular box.
[0084] Thus, when the rectangular frames in the same row include at least two rectangular frames, the distances from the two vertices of the upper frame of the first rectangular frame to the extended line of the upper frame of the second rectangular frame are both less than the first threshold, and the distances from the two vertices of the lower frame of the first rectangular frame to the extended line of the lower frame of the second rectangular frame are both less than the second threshold, and the first rectangular frame and the second rectangular frame are any two rectangular frames in the same row of rectangular frames. When the rectangular frames in the same column include at least two rectangular frames, the distances from the two vertices of the left frame of the third rectangular frame to the extended line of the left frame of the fourth rectangular frame are both less than the third threshold, and the third rectangular frame and the fourth rectangular frame are any two rectangular frames in the same column of rectangular frames.
[0085] In addition, if a rectangular frame and other rectangular frames in the first rectangular image all meet the following conditions, the rectangular frame is treated as a single row or column: the distance between the extended lines of the upper borders of the rectangular frame and other rectangular frames is not less than the first threshold, or the distance between the extended lines of the lower borders is not less than the second threshold. In this case, the rectangular frame is treated as a single row. When the distance between the extended lines of the upper borders is represented by the distance from the two vertices of the upper border of a rectangular frame to the extended line of the upper border of another rectangular frame, the above conditions can also be expressed as: the distance from at least one vertex of the upper border of the rectangular frame to the extended line of the upper border of other rectangular frames is not less than the first threshold, or the distance from at least one vertex of the lower border of the rectangular frame to the extended line of the lower border of other rectangular frames is not less than the second threshold.
[0086] If the distance between a rectangular frame and the extended lines of the left borders of other rectangular frames in the first rectangular image is not less than the third threshold, the rectangular frame is treated as a separate column. When the distance between the extended lines of the left borders is represented by the distance from the two vertices of the left border of a rectangular frame to the extended line of the left border of another rectangular frame, the above condition can also be expressed as: the distance from the two vertices of the left border of the rectangular frame to the extended lines of the left borders of other rectangular frames is not less than the third threshold.
[0087] by Figure 4 For example, the distances between the two vertices of the upper border of rectangular frame 302 and the upper border extension line 402 of rectangular frame 301 are D1 and D2 respectively. Assuming that D1 is greater than the first threshold and D2 is greater than the second threshold, rectangular frame 302 and rectangular frame 301 do not belong to the same row of rectangular frames. Since the distance between the upper border extension line of rectangular frame 301 and the upper border extension lines of all other rectangular frames is greater than the first threshold, there are no other rectangular frames belonging to the same row as rectangular frame 301, and rectangular frame 301 is regarded as a separate row of rectangular frames, which only includes rectangular frame 301.
[0088] The distances between the two vertices of the upper frames of the rectangular frames 303-305 and the upper frame extension line 402 of the rectangular frame 302 are all smaller than the first threshold, and the distances between the two vertices of the lower frames of the rectangular frames 303-305 and the lower frame extension line 403 of the rectangular frame 302 are all smaller than the second threshold, therefore, the rectangular frames 303-305 and the rectangular frame 302 belong to the same row of rectangular frames. The row of rectangular frames includes the rectangular frames 302-305.
[0089] by Figure 5 For example, Figure 5, the distances from the two vertices of the left border of the other rectangular boxes to the extended line 501 of the left border of the rectangular box 301 are all greater than the third threshold, therefore, the rectangular box 301 is taken as a separate column, and the column of rectangular boxes only controls the rectangular box 301. The distances from the two vertices of the left border of the rectangular box 344 to the extended line 502 of the left border of the rectangular box 306 are all less than the third threshold, therefore, the rectangular box 344 and the rectangular box 306 are taken as a column, and the column of rectangular boxes includes the rectangular box 344 and the rectangular box 306. The distances between the two vertices of the left borders of rectangular frame 311, rectangular frame 315, rectangular frame 318, rectangular frame 322, rectangular frame 326, rectangular frame 329, rectangular frame 332, rectangular frame 335, rectangular frame 338, rectangular frame 341, rectangular frame 345, rectangular frame 349 and rectangular frame 353 and the extension line 503 of the left border of rectangular frame 307 are all less than the third threshold, therefore, rectangular frame 307, rectangular frame 311, rectangular frame 315, rectangular frame 318, rectangular frame 322, rectangular frame 326, rectangular frame 329, rectangular frame 332, rectangular frame 335, rectangular frame 338, rectangular frame 341, rectangular frame 345, rectangular frame 349 and rectangular frame 353 are in a column.
[0090] (2) For the left side of the first rectangular image, select the rectangular frames in the first column to form a first rectangular frame set; for the right side of the first rectangular image, select the rectangular frames in the last column to form a second rectangular frame set; for the top side of the first rectangular image, select the rectangular frames in the first row to form a third rectangular frame set; for the bottom side of the first rectangular image, select the rectangular frames in the last row to form a fourth rectangular frame set. Here, the first column is the column closest to the left side of the first rectangular image, and the last column is the column closest to the right side of the first rectangular image. The first row is the row closest to the top side of the first rectangular image, and the last row is the row closest to the bottom side of the first rectangular image.
[0091] Through the above method, a corresponding rectangular frame set can also be generated for each edge of the first rectangular image.
[0092] After the rectangular frame set is generated, the embodiment of the present application determines a target tilt angle according to the tilt angle of the target border of the rectangular frame in the rectangular frame set, where the target border is the frame line in the rectangular frame that is most adjacent to the edge.
[0093] When determining the target tilt angle, the angle between the target border of each rectangular frame in the rectangular frame set and the reference line can be calculated as the tilt angle of the target border of the rectangular frame, and the reference line is a preset horizontal line or vertical line. For example, the reference line can be any side of the first rectangular image. Then, the first statistical value of the angle between the target border of each rectangular frame and the reference line is calculated as the target tilt angle, wherein the first statistical value is the median or average of the angle. When determining the reference point according to the target vertices of each rectangular frame in the rectangular frame set, the coordinates of the target vertices of each rectangular frame in the rectangular frame set can be obtained. Then, the second statistical value of the coordinates of the obtained target vertices is calculated as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertices.
[0094] It should be noted that the number of rectangular frames in the rectangular frame set may be 1 or greater than 1, and therefore, the number of inclination angles of the target frame calculated may be 1 or more. When the number of inclination angles of the target frame calculated is 1, the embodiment of the present application determines the target inclination angle based on 1 inclination angle. For example, the 1 inclination angle is directly used as the target inclination angle. When the number of inclination angles of the target frame calculated is greater than 1, the target inclination angle can be determined based on the first statistical value of these inclination angles.
[0095] For example, for the left side of the first rectangular image, for the first rectangular frame set corresponding to the left side of the first rectangular image, the first target tilt angle can be determined according to the first statistical value of the angle between the left border of each rectangular frame in the first rectangular frame set and the reference line. Then, the coordinates of the upper left corner vertices of each rectangular frame in the first rectangular frame set are obtained, and the second statistical value of the coordinates of these upper left corner vertices is calculated to obtain the coordinates of the first reference point, or the coordinates of the lower left corner vertices of each rectangular frame in the first rectangular frame set are obtained, and the second statistical value of the coordinates of these lower left corner vertices is calculated to obtain the coordinates of the first reference point, or the coordinates of the upper left corner vertex and the lower left corner vertex of each rectangular frame in the first rectangular frame set are obtained, and the second statistical value of the coordinates of these upper left corner vertices and the lower left corner vertices is calculated to obtain the coordinates of the first reference point. Then, a straight line passing through the first reference point and having a tilt angle of the first target tilt angle is generated as the first outer frame line of the borderless table.
[0096] For the right side of the first rectangular image, for the second rectangular frame set corresponding to the right side of the first rectangular image, the second target tilt angle can be determined according to the first statistical value of the angle between the right frame of each rectangular frame in the second rectangular frame set and the reference line. Then, the coordinates of the upper right corner vertices of each rectangular frame in the second rectangular frame set are obtained, and the second statistical values of these upper right corner vertices are calculated to obtain the coordinates of the second reference point, or the coordinates of the lower right corner vertices of each rectangular frame in the second rectangular frame set are obtained, and the second statistical values of these lower right corner vertices are calculated to obtain the coordinates of the second reference point, or the coordinates of the upper right corner vertices and the lower right corner vertices of each rectangular frame in the second rectangular frame set are obtained, and the second statistical values of these upper right corner vertices and the lower right corner vertices are calculated to obtain the coordinates of the second reference point. Then, a straight line passing through the second reference point and having an inclination angle of the second target tilt angle is generated as the second outer frame line of the borderless table.
[0097] For the top edge of the first rectangular image, for the third rectangular frame set corresponding to the top edge of the first rectangular image, the third target tilt angle can be determined according to the first statistical value of the angle between the upper border of each rectangular frame in the third rectangular frame set and the reference line. Then, the coordinates of the upper left corner vertices of each rectangular frame in the third rectangular frame set are obtained, and the second statistical values of these upper left corner vertices are calculated to obtain the coordinates of the third reference point, or the coordinates of the upper right corner vertices of each rectangular frame in the third rectangular frame set are obtained, and the second statistical values of these upper right corner vertices are calculated to obtain the coordinates of the third reference point, or the coordinates of the upper left corner vertices and the upper right corner vertices of each rectangular frame in the third rectangular frame set are obtained, and the second statistical values of these upper left corner vertices and the upper right corner vertices are calculated to obtain the coordinates of the third reference point. Then, a straight line passing through the third reference point and having an inclination angle of the third target tilt angle is generated as the third outer frame line of the borderless table.
[0098] For the bottom edge of the first rectangular image, for the fourth rectangular frame set corresponding to the bottom edge of the first rectangular image, the fourth target tilt angle can be determined according to the first statistical value of the angle between the bottom border of each rectangular frame in the fourth rectangular frame set and the reference line. Then, the coordinates of the lower left corner vertex of each rectangular frame in the fourth rectangular frame set are obtained. The second statistical value of these lower left corner vertices is calculated to obtain the coordinates of the fourth reference point, or the coordinates of the lower right corner vertex of each rectangular frame in the fourth rectangular frame set are obtained. The second statistical value of these lower right corner vertices is calculated to obtain the coordinates of the fourth reference point, or the coordinates of the lower left corner vertex and the lower right corner vertex of each rectangular frame in the fourth rectangular frame set are obtained. The second statistical value of these lower left corner vertices and the lower right corner vertices is calculated to obtain the coordinates of the fourth reference point. Then, a straight line passing through the fourth reference point and having an inclination angle of the fourth target tilt angle is generated as the fourth outer frame line of the borderless table.
[0099] Step 14: Determine four intersection points according to the four outer frame lines of the borderless table.
[0100] Here, according to the four outer frame lines of the borderless table obtained in step 13, the intersection points of these outer frame lines can be obtained, thereby obtaining four intersection points.
[0101] Step 15, performing perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix.
[0102] Here, the four preset target points are the four vertices of the borderless table after perspective transformation, which correspond one-to-one to the four intersection points obtained in step 14. According to the coordinates of the above intersection points and target points, the perspective transformation matrix can be obtained according to the processing method of the prior art.
[0103] Specifically, the four preset target points can be determined in the following manner: any one of the four intersection points is directly used as a preset target point, and then the coordinates of the remaining preset target points are determined according to the coordinates of the any one intersection point and the distance between the any one intersection point and the adjacent intersection point. The following description is made by taking the lower left intersection point as the any one intersection point as an example.
[0104] Assume that the four intersection points obtained in step 14 include:
[0105] The lower left intersection point where the first outer frame line intersects with the fourth outer frame line is assumed to have coordinates (x1, y1);
[0106] The upper left intersection point where the first outer frame line intersects the third outer frame line is assumed to have coordinates (x2, y2);
[0107] The lower right intersection point where the second outer frame line intersects the fourth outer frame line is assumed to have coordinates (x3, y3);
[0108] The upper right intersection point where the second outer frame line intersects the third outer frame line is assumed to have coordinates (x4, y4);
[0109] In the case where the borderless table does not need to be scaled, the lower left intersection can be selected as the first preset target point, that is, the coordinates of the first preset target point corresponding to the lower left intersection are (x1, y1). Then, the first distance L1 between the lower left intersection and the upper left intersection is calculated, and the coordinates of the second preset target point corresponding to the upper left intersection are generated (x1, y1+L1). The second distance L2 between the lower left intersection and the lower right intersection is calculated, and the coordinates of the third preset target point corresponding to the lower right intersection are generated (x1+L2, y1). Finally, the coordinates of the fourth preset target point corresponding to the upper right intersection are obtained (x1+L2, y1+L1).
[0110] When there is no need to scale the borderless table, the first distance and the second distance can be scaled according to the preset scaling ratio α to obtain four preset target points, whose coordinates are: the coordinates of the first preset target point corresponding to the lower left intersection are (x1, y1); the coordinates of the second preset target point corresponding to the upper left intersection are (x1, y1+α*L1); the coordinates of the third preset target point corresponding to the lower right intersection are (x1+α*L2, y1); and the coordinates of the fourth preset target point corresponding to the upper right intersection are (x1+α*L2, y1+α*L1).
[0111] Step 16: Correct the first rectangular image according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0112] Here, according to the perspective transformation matrix, the first rectangular image may be tilt-corrected to obtain a corrected second rectangular image.
[0113] Through the above steps, the embodiment of the present application determines each outer frame line of the borderless table based on the tilt angle and vertex coordinates of the rectangular box obtained by text detection. Since the tilt angles and vertex coordinates of multiple rectangular boxes are comprehensively considered, the embodiment of the present application can improve the accuracy of the feature points of the obtained borderless table image, thereby improving the performance of subsequent image tilt correction processing.
[0114] Based on the above method, the present application also provides a device for implementing the above method. Figure 6 , an embodiment of the present application provides a device for correcting the tilt of a borderless table, comprising:
[0115] An image acquisition module 601 is used to acquire a first rectangular image of a borderless table, wherein the borderless table is filled with text;
[0116] A text detection module 602 is used to perform text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment;
[0117] The outer frame line generating module 603 is used to generate a rectangular frame set for each edge of the first rectangular image, wherein the rectangular frame set includes at least one rectangular frame adjacent to the edge; determine a target tilt angle according to the tilt angle of the target border of the rectangular frame in the rectangular frame set, wherein the target border is the frame line in the rectangular frame that is closest to the edge; determine a reference point according to the target vertices of each rectangular frame in the rectangular frame set, wherein the target vertex is the vertex through which the target border passes; generate a straight line passing through the reference point and having a tilt angle equal to the target tilt angle as an outer frame line of the borderless table;
[0118] The intersection generating module 604 is used to determine four intersections according to the four outer frame lines of the borderless table;
[0119] A matrix obtaining module 605 is used to perform perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix;
[0120] The image correction module 606 is used to correct the first rectangular image according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0121] Through the above modules, the embodiment of the present application can improve the accuracy of the feature points of the obtained borderless table image, thereby improving the performance of subsequent image tilt correction processing.
[0122] Optionally, the outer frame line generating module 603 includes:
[0123] A first determination module is used to determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold;
[0124] A first selection module, configured to select rectangular frames in a first column from the left side of the first rectangular image to form a first rectangular frame set;
[0125] A second selection module is used to select the last column of rectangular frames on the right side of the first rectangular image to form a second rectangular frame set;
[0126] A third selection module, configured to select rectangular frames in a first row according to the top edge of the first rectangular image to form a third rectangular frame set;
[0127] The fourth selection module is used to select the last row of rectangular frames according to the bottom edge of the first rectangular image to form a fourth rectangular frame set.
[0128] Optionally, the outer frame line generating module 603 further includes:
[0129] A first calculation module, used to calculate the angle between the target frame of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line;
[0130] The second calculation module is used to calculate a first statistical value of the angle between the target border of each rectangular frame and the reference line as the target tilt angle, wherein the first statistical value is the median or average value of the angle.
[0131] Optionally, the outer frame line generating module may further include:
[0132] A first acquisition module is used to acquire the coordinates of the target vertex of each rectangular frame in the rectangular frame set;
[0133] The third calculation module is used to calculate the second statistical value of the acquired coordinates of the target vertex as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
[0134] Optionally, the first acquisition module is further used to:
[0135] For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set;
[0136] For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set;
[0137] For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set;
[0138] With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
[0139] It should be noted that the various devices / systems provided in the above embodiments are devices corresponding to the above-mentioned tilt correction method for borderless tables, and the implementation methods in the above-mentioned embodiments are applicable to the embodiments of the device, and can also achieve the same technical effects. The above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment will not be specifically described here.
[0140] Please refer to Figure 7 The embodiment of the present application also provides a hardware structure block diagram of a tilt correction device for a borderless table, such as Figure 7 As shown, the tilt correction device 700 of the borderless table includes:
[0141] Processor 702; and
[0142] a memory 704 in which computer program instructions are stored,
[0143] When the computer program instructions are executed by the processor, the processor 702 is caused to perform the following steps:
[0144] Obtaining a first rectangular image of a borderless table, wherein the borderless table is filled with text;
[0145] Performing text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment;
[0146] For each edge of the first rectangular image, a rectangular frame set is generated respectively, the rectangular frame set includes at least one rectangular frame adjacent to the edge; according to the inclination angle of the target frame of the rectangular frame in the rectangular frame set, a target inclination angle is determined, the target frame is the frame of the rectangular frame that is most adjacent to the edge; according to the target vertices of each rectangular frame in the rectangular frame set, a reference point is determined, the target vertex is the vertex through which the target frame passes; a straight line passing through the reference point and having an inclination angle of the target inclination angle is generated as an outer frame line of the borderless table;
[0147] According to the four outer frame lines of the borderless table, four intersection points are determined;
[0148] Perform perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix;
[0149] The first rectangular image is corrected according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0150] It should be noted that the various systems provided in the above embodiments are devices corresponding to the above-mentioned tilt correction method for borderless tables, and the implementation methods in the above-mentioned embodiments are applicable to the embodiments of the device, and can also achieve the same technical effects. The above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment will not be specifically described here.
[0151] Furthermore, if Figure 7 As shown, the borderless table tilt correction device 700 also includes a network interface 701 , an input device 703 , a hard disk 705 , and a display device 706 .
[0152] The above-mentioned interfaces and devices can be interconnected through a bus architecture. The bus architecture can be a bus and bridge that can include any number of interconnected buses. Specifically, one or more central processing units (CPUs) and / or graphics processing units (GPUs) represented by processor 702, and various circuits of one or more memories represented by memory 704 are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together. It can be understood that the bus architecture is used to achieve connection and communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are all well known in the art, so they are not described in detail herein.
[0153] The network interface 701 can be connected to a network (such as the Internet, a local area network, etc.), receive data such as the first rectangular image of the borderless table from the network, and can save the received data in the hard disk 705.
[0154] The input device 703 can receive various instructions input by the operator and send them to the processor 702 for execution. The input device 703 can include a keyboard or a pointing device (for example, a mouse, a trackball, a touch pad or a touch screen, etc.).
[0155] The display device 706 can display the result obtained by the processor 702 executing the instructions, such as displaying a corrected image.
[0156] The memory 704 is used to store programs and data necessary for the operation of the operating system, as well as data such as intermediate results during the calculation process of the processor 702.
[0157] It is understood that the memory 704 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. The memory 704 of the apparatus and method described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0158] In some implementations, the memory 704 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 7041 and an application program 7042 .
[0159] The operating system 7041 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 7042 includes various application programs, such as a browser, etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 7042.
[0160] The method disclosed in the above embodiment of the present application can be applied to the processor 702, or implemented by the processor 702. The processor 702 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 702. The above processor 702 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 704, and the processor 702 reads the information in the memory 704 and completes the steps of the above method in combination with its hardware.
[0161] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.
[0162] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0163] Specifically, when the computer program is executed by the processor 702, the following steps may also be implemented:
[0164] Determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold;
[0165] For the left side of the first rectangular image, select the rectangular frames in the first column to form a first rectangular frame set; for the right side of the first rectangular image, select the rectangular frames in the last column to form a second rectangular frame set; for the top side of the first rectangular image, select the rectangular frames in the first row to form a third rectangular frame set; for the bottom side of the first rectangular image, select the rectangular frames in the last row to form a fourth rectangular frame set.
[0166] Specifically, when the computer program is executed by the processor 702, the following steps may also be implemented:
[0167] Calculate the angle between the target border of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line;
[0168] A first statistical value of an angle between a target frame of each rectangular frame and a reference line is calculated as the target tilt angle, wherein the first statistical value is a median or an average value of the angle.
[0169] Specifically, when the computer program is executed by the processor 702, the following steps may also be implemented:
[0170] Get the coordinates of the target vertices of each rectangular box in the rectangular box set;
[0171] A second statistical value of the acquired coordinates of the target vertex is calculated as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
[0172] Specifically, when the computer program is executed by the processor 702, the following steps may also be implemented:
[0173] For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set;
[0174] For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set;
[0175] For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set;
[0176] With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
[0177] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0178] In some embodiments of the present application, a computer-readable storage medium is further provided, on which a program is stored, and when the program is executed by a processor, the following steps are implemented:
[0179] Obtaining a first rectangular image of a borderless table, wherein the borderless table is filled with text;
[0180] Performing text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment;
[0181] For each edge of the first rectangular image, a rectangular frame set is generated respectively, the rectangular frame set includes at least one rectangular frame adjacent to the edge; according to the inclination angle of the target frame of the rectangular frame in the rectangular frame set, a target inclination angle is determined, the target frame is the frame of the rectangular frame that is most adjacent to the edge; according to the target vertices of each rectangular frame in the rectangular frame set, a reference point is determined, the target vertex is the vertex through which the target frame passes; a straight line passing through the reference point and having an inclination angle of the target inclination angle is generated as an outer frame line of the borderless table;
[0182] According to the four outer frame lines of the borderless table, four intersection points are determined;
[0183] Perform perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix;
[0184] The first rectangular image is corrected according to the perspective transformation matrix to obtain a corrected second rectangular image.
[0185] When the program is executed by the processor, it can implement all the implementation methods of the above-mentioned borderless table tilt correction method and achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0186] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0188] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0189] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0191] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.
[0192] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for correcting the tilt of a borderless table, characterized in that: include: · Obtaining a first rectangular image of a borderless table, wherein the borderless table is filled with text; Performing text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment; For each edge of the first rectangular image, a rectangular frame set is generated respectively, the rectangular frame set includes at least one rectangular frame adjacent to the edge; a target tilt angle is determined according to the tilt angle of the target border of the rectangular frame in the rectangular frame set, the target border is the border of the rectangular frame that is most adjacent to the edge; a reference point is determined according to the target vertex of the rectangular frame in the rectangular frame set, the target vertex is the vertex through which the target border passes; a straight line passing through the reference point and having an tilt angle of the target tilt angle is generated as an outer frame line of the borderless table; According to the four outer frame lines of the borderless table, four intersection points are determined; Perform perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix; The first rectangular image is corrected according to the perspective transformation matrix to obtain a corrected second rectangular image.
2. The method according to claim 1, characterized in that For each edge of the first rectangular image, a rectangular frame set is generated, including: Determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold; For the left side of the first rectangular image, select the rectangular frames in the first column to form a first rectangular frame set; for the right side of the first rectangular image, select the rectangular frames in the last column to form a second rectangular frame set; for the top side of the first rectangular image, select the rectangular frames in the first row to form a third rectangular frame set; for the bottom side of the first rectangular image, select the rectangular frames in the last row to form a fourth rectangular frame set.
3. The method according to claim 2, characterized in that According to the inclination angle of the target border of the rectangular frame in the rectangular frame set, a target inclination angle is determined, including: Calculate the angle between the target border of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line; A first statistical value of an angle between a target frame of each rectangular frame and a reference line is calculated as the target tilt angle, wherein the first statistical value is a median or an average value of the angle.
4. The method according to claim 2, characterized in that According to the target vertices of each rectangular box in the rectangular box set, a reference point is determined, including: Get the coordinates of the target vertices of each rectangular box in the rectangular box set; A second statistical value of the acquired coordinates of the target vertex is calculated as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
5. The method according to claim 4, characterized in that Get the coordinates of the target vertices of each rectangular box in the rectangular box set, including: For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set; For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set; For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set; With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
6. A device for correcting the tilt of a borderless table, characterized in that: include: An image acquisition module, used to acquire a first rectangular image of a borderless table, wherein the borderless table is filled with text; A text detection module, used to perform text detection on the first rectangular image to obtain a text segment in the first rectangular image and a rectangular frame circumscribing the text segment; The outer frame line generation module is used to generate a rectangular frame set for each edge of the first rectangular image, wherein the rectangular frame set includes at least one rectangular frame adjacent to the edge; determine a target tilt angle according to the tilt angle of the target border of the rectangular frame in the rectangular frame set, wherein the target border is the frame line in the rectangular frame that is closest to the edge; determine a reference point according to the target vertices of each rectangular frame in the rectangular frame set, wherein the target vertex is the vertex through which the target border passes; generate a straight line passing through the reference point and having an tilt angle equal to the target tilt angle as an outer frame line of the borderless table; An intersection point generation module, used to determine four intersection points according to the four outer frame lines of the borderless table; A matrix acquisition module, used for performing perspective transformation according to the four intersection points and four preset target points to obtain a perspective transformation matrix; The image correction module is used to correct the first rectangular image according to the perspective transformation matrix to obtain a corrected second rectangular image.
7. The device according to claim 6, characterized in that The outer frame line generation module includes: A first determination module is used to determine the row and column to which the rectangular frames in the first rectangular image belong, wherein, in the rectangular frames in the same row, the distance between the extended lines of the upper borders of any two rectangular frames is less than a first threshold, and the distance between the extended lines of the lower borders is less than a second threshold; in the rectangular frames in the same column, the distance between the extended lines of the left borders of any two rectangular frames is less than a third threshold; A first selection module, configured to select rectangular frames in a first column from the left side of the first rectangular image to form a first rectangular frame set; A second selection module is used to select the last column of rectangular frames on the right side of the first rectangular image to form a second rectangular frame set; A third selection module, configured to select rectangular frames in a first row according to the top edge of the first rectangular image to form a third rectangular frame set; The fourth selection module is used to select the last row of rectangular frames according to the bottom edge of the first rectangular image to form a fourth rectangular frame set.
8. The device according to claim 7, characterized in that The outer frame line generation module also includes: A first calculation module, used to calculate the angle between the target frame of each rectangular frame in the rectangular frame set and a reference line, where the reference line is a preset horizontal line or vertical line; The second calculation module is used to calculate a first statistical value of the angle between the target border of each rectangular frame and the reference line as the target tilt angle, wherein the first statistical value is the median or average value of the angle.
9. The device according to claim 7, characterized in that The outer frame line generation module also includes: A first acquisition module is used to acquire the coordinates of the target vertices of each rectangular frame in the rectangular frame set; The third calculation module is used to calculate the second statistical value of the acquired coordinates of the target vertex as the coordinates of the reference point, wherein the second statistical value is the median or average of the coordinates of the target vertex.
10. The device according to claim 9, characterized in that The first acquisition module is further used for: For the left side of the first rectangular image, obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first rectangular set; For the right side of the first rectangular image, obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second rectangular set; For the top edge of the first rectangular image, obtaining the coordinates of the upper left corner vertex and / or the upper right corner vertex of each rectangular frame in the third rectangular set; With respect to the bottom edge of the first rectangular image, the coordinates of the lower left corner vertex and / or the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained.
11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for correcting the tilt of a borderless table according to any one of claims 1 to 5 are implemented.
