Inclination correction method and inclination correction device for frameless table, and program

The method enhances frameless table image tilt correction by using text detection and perspective transformation to improve feature point accuracy and image processing performance.

JP2024169402AActive Publication Date: 2024-12-05RICOH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2024084085
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-24
Filing Date
2024-05-23
Publication Date
2024-12-05
Estimated Expiration
2044-05-23

AI Technical Summary

Technical Problem

Existing methods struggle to accurately correct the tilt of frameless table images due to the difficulty in obtaining feature points, leading to low accuracy in subsequent image processing and table identification.

Method used

A method involving text detection to identify rectangular frames, determining inclination angles and vertices, generating outer frame lines, and performing perspective transformation to correct the tilt of frameless table images.

Benefits of technology

Improves the accuracy of feature point detection and tilt correction for frameless table images, enhancing the performance of subsequent image processing and identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169402000001_ABST
    Figure 2024169402000001_ABST
Patent Text Reader

Abstract

To provide an inclination correction method and an inclination correction device for a frameless table, and a storage medium.SOLUTION: An inclination correction method and an inclination correction device for a frameless table provided in an embodiment of the present invention fix an outer frame line of a frameless table on the basis of the inclination angle and vertex coordinates of a rectangular frame obtained by text detection. The inclination correction method and inclination correction device for a frameless table of the embodiment of the prevent invention comprehensively consider the inclination angles and vertex coordinates of a plurality of rectangular frames, and thereby can improve the accuracy of the feature points of an obtained image of the frameless table, and can improve the efficiency of subsequent image inclination correction processing.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of image processing technology, and specifically to a method and device for correcting tilt of a frameless table, and a storage medium. [Background technology]

[0002] In document processing, it is often necessary to identify text printed on paper. It is common to capture the text by photographing the paper and obtaining an image containing the text, then recognizing the image with OCR (Optical Character Recognition). For example, an image containing a table requires table structure analysis and text recognition on the image to obtain the table cells and the text content within the cells.

[0003] When capturing an image containing text, distortion is likely to occur. Conventionally, a process is first performed to correct the image tilt through a perspective transformation process. For an image of a table with a complete frame, feature points for the perspective transformation process can be easily obtained. In practical applications, some tables are images of tables without frames, that is, vertical and horizontal lines in the table are omitted. For an image of a table without a frame, it is difficult to obtain feature points for the perspective transformation process, or the obtained feature points are not accurate, which is disadvantageous for the subsequent image tilt correction and table identification. Summary of the Invention [Problem to be solved by the invention]

[0004] At least one embodiment of the present invention provides a frameless table tilt correction method, a tilt correction device and a storage medium, which can improve the accuracy of feature points of the captured frameless table image and improve the performance of the image tilt correction processing. [Means for solving the problem]

[0005] In order to solve the above technical problem, an embodiment of the present invention includes the steps of: obtaining a first rectangular image of a frameless table including text; performing text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; generating a rectangular frame set including at least one rectangular frame near each side of the first rectangular image; determining a target tilt angle based on a tilt angle of a target frame side of a rectangular frame in the rectangular frame set; determining a reference point based on a target vertex of a rectangular frame in the rectangular frame set; and determining a target tilt angle based on a target vertex of a rectangular frame that passes through the reference point and satisfies the target tilt angle. the target frame side is a frame side closest to the side in the rectangular frame, and the target vertex is a vertex through which the target frame side passes; based on the four outer frame lines of the frameless table, determining four intersections; performing perspective transformation based on the four intersections and four predetermined target points to obtain a perspective transformation matrix; and performing correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image.

[0006] Preferably, the step of generating a rectangular frame set for each side of the first rectangular image includes the steps of identifying a matrix to which a rectangular frame in the first rectangular image belongs, wherein, for rectangular frames in the same row, the distance between the upper frame extension lines of any two rectangular frames is smaller than a first threshold and the distance between the lower frame extension lines is smaller than a second threshold, and for rectangular frames in the same column, the distance between the left frame extension lines of any two rectangular frames is smaller than a third threshold, and selecting a rectangular frame in the first column for the left side of the first rectangular image to construct a first rectangular frame set, selecting a rectangular frame in the last column for the right side of the first rectangular image to construct a second rectangular frame set, selecting a rectangular frame in the first row for the top side of the first rectangular image to construct a third rectangular frame set, and selecting a rectangular frame in the last row for the bottom side of the first rectangular image to construct a fourth rectangular frame set.

[0007] In addition, preferably, the step of determining a target inclination angle based on the inclination angle of a target frame side of a rectangular frame in the rectangular frame set includes a step of calculating an angle of the target frame side of each rectangular frame in the rectangular frame set relative to a reference line, the reference line being a predetermined horizontal line or vertical line, and a step of calculating a first statistical value of the angle of each rectangular frame relative to the reference line of the target frame as the target inclination angle, the first statistical value being a median or average value of the angles.

[0008] Also, preferably, the step of determining a reference point based on a target vertex of a rectangular frame in the rectangular frame set includes the steps of acquiring coordinates of the target vertex of each rectangular frame in the rectangular frame set, and calculating a second statistical value of the acquired coordinates of the target vertex as the coordinate of the reference point, wherein the second statistical value is the median or average value of the coordinates of the target vertices.

[0009] In addition, preferably, the step of acquiring the coordinates of the target vertices of each rectangular frame in the rectangular frame set includes the steps of acquiring the coordinates of the upper left and / or lower left vertex of each rectangular frame in the first rectangular set with respect to the left side of the first rectangular image, acquiring the coordinates of the upper right and / or lower right vertex of each rectangular frame in the second rectangular set with respect to the right side of the first rectangular image, acquiring the coordinates of the upper left and / or upper right vertex of each rectangular frame in the third rectangular set with respect to the top side of the first rectangular image, and acquiring the coordinates of the lower left and / or lower right vertex of each rectangular frame in the fourth rectangular set with respect to the bottom side of the first rectangular image.

[0010] Also, an embodiment of the present invention includes an image acquisition module that acquires a first rectangular image of a frameless table including text; a text detection module that performs text detection on the first rectangular image and acquires a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; a rectangular frame set that generates, for each side of the first rectangular image, at least one rectangular frame in a vicinity of the side; determines a target tilt angle based on a tilt angle of a target frame side of a rectangular frame in the rectangular frame set; determines a reference point based on a target vertex of a rectangular frame in the rectangular frame set; and draws a straight line that passes through the reference point and has a tilt angle of the target tilt angle on the frameless table. the target frame side is a frame side closest to the side in the rectangular frame, and the target vertex is a vertex through which the target frame side passes; an intersection generation module that determines four intersections based on four outer frame lines of the frameless table; a matrix acquisition module that performs perspective transformation based on the four intersections and four predetermined target points to acquire a perspective transformation matrix; and an image correction module that performs correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image.

[0011] Preferably, the outer border generation module includes a first identification module that identifies a matrix to which a rectangular frame in the first rectangular image belongs, where in rectangular frames in the same row, the distance between the upper frame extension lines of any two rectangular frames is smaller than a first threshold and the distance between the lower frame extension lines is smaller than a second threshold, and in rectangular frames in the same column, the distance between the left frame extension lines of any two rectangular frames is smaller than a third threshold; a first selection module that selects a rectangular frame in the first column for the left side of the first rectangular image and constitutes a first rectangular frame set; a second selection module that selects a rectangular frame in the last column for the right side of the first rectangular image and constitutes a second rectangular frame set; a third selection module that selects a rectangular frame in the first row for the top side of the first rectangular image and constitutes a third rectangular frame set; and a fourth selection module that selects a rectangular frame in the last row for the bottom side of the first rectangular image and constitutes a fourth rectangular frame set.

[0012] In addition, preferably, the outer frame line generation module includes a first calculation module that calculates an angle of a target frame side of each rectangular frame in the rectangular frame set relative to a reference line, the reference line being a predetermined horizontal or vertical line, and a second calculation module that calculates a first statistical value of the angle of each rectangular frame relative to the reference line of the target frame as the target inclination angle, the first statistical value being a median or average of the angles.

[0013] In addition, preferably, the outer frame line generation module further includes a first acquisition module that acquires coordinates of target vertices of each rectangular frame in the rectangular frame set, and a third calculation module that calculates a second statistical value of the acquired coordinates of the target vertices as the coordinate of the reference point, wherein the second statistical value is the median or average value of the coordinates of the target vertices.

[0014] In addition, preferably, the first acquisition module acquires the coordinates of the upper left and / or lower left vertex of each rectangular frame in the first rectangular collection with respect to the left side of the first rectangular image, acquires the coordinates of the upper right and / or lower right vertex of each rectangular frame in the second rectangular collection with respect to the right side of the first rectangular image, acquires the coordinates of the upper left and / or upper right vertex of each rectangular frame in the third rectangular collection with respect to the top side of the first rectangular image, and acquires the coordinates of the lower left and / or lower right vertex of each rectangular frame in the fourth rectangular collection with respect to the bottom side of the first rectangular image.

[0015] Furthermore, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, the computer program being executed by a processor to realize steps of the above-mentioned frameless table tilt correction method.

[0016] Compared with the prior art, the frameless table skew correction method and apparatus according to the embodiment of the present invention determines each outer border of the frameless table based on the skew angle and vertex coordinates of the rectangular frame obtained by text detection, and comprehensively takes into account the skew angles and vertex coordinates of multiple rectangular frames, thereby improving the accuracy of the feature points of the frameless table image and improving the performance of the subsequent image skew correction process. [Brief description of the drawings]

[0017] Various other benefits and advantages will become apparent to those skilled in the art from the detailed description of the preferred embodiment below. The accompanying drawings are used only to illustrate the preferred embodiment and are not intended to limit the present invention. The same reference numerals are used throughout the drawings to refer to the same parts. [Figure 1] FIG. 1 is a flowchart showing a method for correcting tilt of a frameless table according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of a first rectangular image including a frameless table according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing an example of a rectangular frame obtained by performing text detection on the first rectangular image. [Figure 4] FIG. 4 is a diagram showing an example of specifying rectangular frames in the same row in the embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of specifying rectangular frames in the same column in the embodiment of the present invention. [Figure 6] FIG. 6 is a block diagram showing the configuration of a frameless table tilt correction device according to an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing another configuration of a tilt correction device for a frameless table according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] In the following, in order to clarify the problems, configurations and effects that the present invention is intended to solve, a detailed description will be given in conjunction with drawings and specific embodiments. In the following description, specific details of specific arrangements and components are provided only to assist in a full understanding of the embodiments of the present invention. Therefore, it is obvious to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, for the sake of clarity and conciseness, descriptions of known functions and configurations will be omitted.

[0019] References throughout the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of the present application. Thus, "in one embodiment" or "in one embodiment" appearing in various places in the specification do not necessarily refer to the same embodiment. Alternatively, these particular features, structures, or characteristics may be combined in any suitable manner with one or more embodiments. The terms "first," "second," and the like in the specification and claims of the present application are used to distinguish between similar objects, and are not necessarily used to describe a particular sequence or order. It should be understood that such used data are interchangeable in appropriate circumstances, whereby the embodiments of the present invention described herein may, for example, be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to the explicitly recited steps or units, but may include other steps or units not explicitly recited or inherent to such process, method, product, or apparatus. In the specification and claims, "and / or" means at least one of the objects connected.

[0020] In each embodiment of the present invention, the magnitude of the numbers of the following processes does not mean the order of execution. The execution order of each process is determined by its function and inherent logic, and does not limit the implementation process of the embodiment of the present invention.

[0021] The following description provides examples and is not intended to limit the scope, applicability, or configuration of the claims. Changes may be made in the function or arrangement of the elements discussed without departing from the spirit and scope of the invention. Procedures and components of the various examples may be omitted, substituted, or added as appropriate. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described in some examples may be combined in other examples.

[0022] In order to improve the accuracy of the feature points of the acquired frameless table image, improve the performance of the image skew correction processing, and further improve the accuracy of identifying the frameless table, an embodiment of the present invention provides a frameless table skew correction method, which, as shown in Figure 1, includes the following steps:

[0023] In step 11, a first rectangular image of a frameless table containing text is obtained, where frameless table means that the table has no vertical or horizontal lines. Table 1 shows an example of a frameless table.

[0024] [Table 1] In an embodiment of the present invention, an image including a frameless table is obtained by photographing a sheet of paper on which the frameless table is printed. Usually, the photographed image is a rectangular image having four sides (left, top, right, bottom), namely, a left side, a top side, a right side, and a bottom side. Here, the left side faces the right side, and the top side faces the bottom side. FIG. 2 is an example of a first rectangular image photographed of the frameless table shown in Table 1. The first rectangular image 20 shown in FIG. 2 has four sides, namely, a left side 201, a top side 202, a right side 203, and a bottom side 204. The frameless table in FIG. 2 is slanted.

[0025] In step 12, the method for correcting skew of a frameless table according to the embodiment of the present invention performs text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion.

[0026] Here, the text can be detected from the first rectangular image by OCR technology, which is a conventional technology. The OCR technology identifies characters in an image, obtains a text portion consisting of at least one character, and generates a rectangular frame circumscribing the text portion. FIG. 3 shows the detection result of performing text detection on the first rectangular image of the frameless table shown in FIG. 2, and the first rectangular image includes 56 rectangular frames, 301 to 356. Since the captured image has a certain distortion, the sides 201 to 204 of the first rectangular image are not strictly parallel or perpendicular to the four sides of the rectangular frames shown in FIG. 3. Each rectangular frame has four frame sides, namely, a left frame side, an upper frame side, a right frame side, and a lower frame side. Each rectangular frame has four vertices, namely, an upper left corner vertex, an upper right corner vertex, a lower left corner vertex, and a lower right corner vertex. The two vertices of the left frame side are the upper left corner vertex and the lower left frame vertex. The two vertices of the top frame are the upper left corner vertex and the upper right corner vertex. The two vertices of the bottom frame edge are the lower-left corner vertex and the lower-right corner vertex.

[0027] In step 13, a rectangular frame set is generated that includes at least one rectangular frame near each side of the first rectangular image; a target inclination angle is identified based on the inclination angle of a target frame side that is the frame side closest to the side in a rectangular frame in the rectangular frame set; a reference point is identified based on a target vertex that is a vertex through which the target frame side passes in a rectangular frame in the rectangular frame set; and a straight line that passes through the reference point and has a inclination angle equal to the target inclination angle is generated as the outer frame line of the frameless table.

[0028] Here, a rectangular frame set is generated for each side of the first rectangular image, the rectangular frame set including at least one rectangular frame in the vicinity of the side. In the embodiment of the present invention, there are several methods for generating the rectangular frame set. The specific methods will be described below.

[0029] As a method for generating a set of rectangular frames, a first method selects, for each side of a first rectangular image, a predetermined number of rectangular frames that are closest to that side, and configures a set of rectangular frames.

[0030] Specifically, first, the distance from a predetermined vertex of each rectangular frame to the corresponding side is calculated. For example, for the left side of the first rectangular image, the distance from the upper left vertex or the lower left vertex of the rectangular frame is calculated. Also, for the top side of the first rectangular image, the distance from the upper left vertex or the upper right vertex of the rectangular frame is calculated. Furthermore, for the right side of the first rectangular image, the distance from the upper right vertex or the lower right vertex of the rectangular frame is calculated, and for the bottom side of the first rectangular image, the distance from the lower left vertex or the lower right vertex of the rectangular frame is calculated.

[0031] Then, a predetermined number of rectangular frames are selected in ascending order of distance to form a rectangular frame set. The predetermined number of rectangular frames corresponding to each side of the first rectangular image may be the same. For example, for each side of the first rectangular image, three rectangular frames having the shortest distance from the side are selected to form one rectangular frame set. Taking FIG. 3 as an example, the three closest rectangular frames, for example, rectangular frame 301, rectangular frame 306, and rectangular frame 344, are selected to form a rectangular frame set for the left side 201 of the first rectangular image. The three closest rectangular frames, for example, rectangular frame 301, rectangular frame 302, and rectangular frame 303, are selected to form a rectangular frame set for the top side 202 of the first rectangular image. The three closest rectangular frames, for example, rectangular frame 305, rectangular frame 310, and rectangular frame 314, are selected to form a rectangular frame set for the right side 203 of the first rectangular image. The three closest rectangular frames, for example, rectangular frame 354, 355, and 356, are selected to form a rectangular frame set for the bottom side 204 of the first rectangular image.

[0032] In addition, in the embodiment of the present invention, the predetermined number corresponding to each side of the first rectangular image may be different or may be partially the same. For example, four rectangular frames with the smallest distance are selected in order from smallest to largest distance to the left side of the first rectangular image to form a rectangular frame set. Similarly, three rectangular frames with the smallest distance are selected to form a rectangular frame set to the top side of the first rectangular image. Furthermore, four rectangular frames with the smallest distance are selected to form a rectangular frame set to the right side of the first rectangular image. Furthermore, three rectangular frames with the smallest distance are selected to form a rectangular frame set to the bottom side of the first rectangular image.

[0033] As a method for generating a rectangular frame set, the second method selects, for each side of the first rectangular image, a rectangular frame whose distance to that side is equal to or less than a preset distance threshold, and constitutes a rectangular frame set. Specifically, first, the distance from a specified vertex of each rectangular frame to that side is calculated. See above for the calculation method. Then, a corresponding distance threshold is set for each side of the first rectangular image. The distance thresholds corresponding to each side may be the same, partially the same, or completely different. Then, a rectangular frame whose distance to that side is equal to or less than the corresponding distance threshold is selected, and a rectangular frame set is constituted.

[0034] The third method for generating a rectangular frame set is as follows.

[0035] (1) First, the matrix to which the rectangular frame in the first rectangular image belongs is identified. That is, the rectangular frames that belong to the same row and the same column in the first rectangular image are identified. Since the image of the frameless table may be distorted, when identifying the matrix, the matrix relationship is identified from the extension lines of the frame sides of the rectangular frame.

[0036] Specifically, for any two rectangular frames in the same row, the distance between the extension lines of the upper frame side is smaller than the first threshold value, and the distance between the extension lines of the lower frame side is smaller than the second threshold value. Also, for any two rectangular frames in the same column, the distance between the extension lines of the left frame side is smaller than the third threshold value. The first, second, and third threshold values ​​may be the same or different. Specifically, these threshold values ​​can be set based on parameters such as character size, character spacing, and line spacing.

[0037] Considering that corresponding sides of rectangular frames may not be parallel, the distance between the extension lines of the top sides is expressed as the distance from the two vertices of the top side of one rectangular frame to the extension line of the top side of the other rectangular frame. The distance between the extension lines of the bottom sides is expressed as the distance from the two vertices of the bottom side of one rectangular frame to the extension line of the bottom side of the other rectangular frame. The distance between the extension lines of the left sides is expressed as the distance from the two vertices of the left side of one rectangular frame to the extension line of the left side of the other rectangular frame.

[0038] In this way, when there are at least two rectangular frames belonging to the same row, the distances from the two vertices of the top frame side of the first rectangular frame to the extension line of the top frame side of the second rectangular frame are all smaller than the first threshold value, and the distances from the two vertices of the bottom frame side of the first rectangular frame to the extension line of the bottom frame side of the second rectangular frame are all smaller than the second threshold value. The first rectangular frame and the second rectangular frame are any two rectangular frames belonging to the same row. Also, when there are at least two rectangular frames belonging to the same column, the distances from the two vertices of the left frame side of the third rectangular frame to the extension line of the left frame side of the fourth rectangular frame are all smaller than the third threshold value. The third rectangular frame and the fourth rectangular frame are any two rectangular frames belonging to the same column.

[0039] Note that one rectangular frame is considered to be in a row or column by itself when it satisfies the condition that the distance between the extended lines of the upper frame sides is equal to or greater than a first threshold value, or the distance between the extended lines of the lower frame sides is equal to or greater than a second threshold value, relative to the other rectangular frame in the first rectangular image. In addition, when the distance between the extended lines of the upper frame sides is expressed using the distance from two vertices of the upper frame side of one rectangular frame to the extended lines of the upper frame side of the other rectangular frame, the above condition can be expressed by the distance from at least one vertex of the upper frame side of one rectangular frame to the extended line of the upper frame side of the other rectangular frame being equal to or greater than a first threshold value, or the distance from at least one vertex of the lower frame side of one rectangular frame to the extended line of the lower frame side of the other rectangular frame being equal to or greater than a second threshold value.

[0040] When the distances between the extension lines of the left frame side of one rectangular frame and the extension lines of the left frame side of the other rectangular frame in the first rectangular image are all equal to or greater than a third threshold, the above condition may be expressed by the distances between the two vertices of the left frame side of one rectangular frame and the extension lines of the left frame side of the other rectangular frame being equal to or greater than a third threshold when the distances between the extension lines of the left frame side of the one rectangular frame and the extension lines of the left frame side of the other rectangular frame are expressed by the distances between the two vertices of the left frame side of the one rectangular frame and the extension lines of the left frame side of the other rectangular frame being equal to or greater than a third threshold.

[0041] Take Fig. 4 as an example. Here, the distances from the two vertices of the top frame side of rectangular frame 302 to extension line 402 of the top frame side of rectangular frame 301 are D1 and D2, respectively. Since D1 is greater than the first threshold and D2 is greater than the second threshold, rectangular frame 302 and rectangular frame 301 do not belong to the same row. Also, since the distances between the extension line of the top frame side of rectangular frame 301 and the extension lines of the top frame sides of other rectangular frames are all greater than the first threshold, there are no other rectangular frames that belong to the same row as rectangular frame 301, and rectangular frame 301 belongs to a row by itself. In other words, the row includes only rectangular frame 301.

[0042] The distances from the two vertices of the top frame side of rectangular frames 303-305 to an extension line 402 of the top frame side of rectangular frame 302 are all smaller than a first threshold value, and the distance from the two vertices of the bottom frame side of rectangular frames 303-305 to an extension line 403 of the bottom frame side of rectangular frame 302 is also smaller than a second threshold value. Therefore, rectangular frames 303-305 and rectangular frame 302 are rectangular frames that belong to the same column. That is, the rectangular frames belonging to that row include rectangular frames 302 to 305.

[0043] Also, take FIG. 5 as an example. FIG. 5 shows extension line 501 of the left frame side of rectangular frame 301. Since the distances from the two vertices of the left frame side of rectangular frames other than rectangular frame 301 to extension line 501 of the left frame side of rectangular frame 301 are all greater than the third threshold, rectangular frame 301 forms a column by itself. The rectangular frames in this column control only rectangular frame 301. In contrast, the distances from the two vertices of the left frame side of rectangular frame 344 to extension line 502 of the left frame side of rectangular frame 306 are all smaller than the third threshold. Therefore, rectangular frame 344 and rectangular frame 306 belong to the same column. In other words, the rectangular frames that belong to this column include rectangular frame 344 and rectangular frame 306. In addition, the distances of the two vertices of the left frame side of rectangular frame 311, rectangular frame 315, rectangular frame 318, rectangular frame 322, rectangular frame 326, rectangular frame 329, rectangular frame 332, rectangular frame 335, rectangular frame 338, rectangular frame 341, rectangular frame 345, rectangular frame 349, and rectangular frame 353 to the extension line 503 of the left frame side of rectangular frame 307 are all less than the third threshold, so rectangular frame 307, rectangular frame 311, rectangular frame 315, rectangular frame 318, rectangular frame 322, rectangular frame 326, rectangular frame 329, rectangular frame 332, rectangular frame 335, rectangular frame 338, rectangular frame 341, rectangular frame 345, rectangular frame 349, and rectangular frame 353 belong to one column.

[0044] (2) For the left side of the first rectangular image, a rectangular frame belonging to the first column is selected to form a first rectangular frame set. For the right side of the first rectangular image, a rectangular frame belonging to the last column is selected to form a second rectangular frame set. For the top side of the first rectangular image, a rectangular frame belonging to the first row is selected to form a third rectangular frame set. Finally, for the bottom side of the first rectangular image, a rectangular frame belonging to the last row is selected to form a fourth rectangular frame set. Here, the first column is the column closest to the left side of the first rectangular image, and the last column is the column closest to the right side of the first rectangular image. Also, the first row is the row closest to the top side of the first rectangular image, and the last row is the row closest to the bottom side of the first rectangular image.

[0045] By using the above method, a set of rectangular frames corresponding to each side of the first rectangular image can be generated.

[0046] After generating the rectangular frame set, the embodiment of the present invention determines the target slant angle based on the slant angle of the target frame side, which is the frame side closest to the said side in a rectangular frame in the rectangular frame set.

[0047] The target tilt angle is specified by calculating the tilt angle of the target frame side of each rectangular frame in the rectangular frame set with respect to a reference line, and setting the tilt angle of the target frame side of the rectangular frame. The reference line is a preset horizontal line or vertical line. For example, any side of the first rectangular image is set as the reference line. Then, a first statistical value, which is the median or average value of the angles of the target frame side of each rectangular frame with respect to the reference line, is calculated and set as the target tilt angle. Also, when determining the reference point from the target vertex of each rectangular frame in the rectangular frame set, the coordinates of the target vertex of each rectangular frame in the rectangular frame set are acquired. Then, a second statistical value, which is the median or average value of the coordinates of the acquired target vertices, is calculated and set as the coordinate of the reference point.

[0048] Here, the number of rectangular frames in the rectangular frame set is one or more. Therefore, the number of calculated inclination angles of the target frame side is one or more. When the number of calculated inclination angles of the target frame side is one, the embodiment of the present invention determines the target inclination angle based on this one inclination angle. For example, this one inclination angle is used as the target inclination angle as it is. Also, when the number of calculated inclination angles of the target frame side is multiple, the target inclination angle is determined based on the first statistical value of these inclination angles.

[0049] For example, the first target tilt angle is determined based on a first statistical value of the angle of the left frame side of each rectangular frame in the first rectangular frame set corresponding to the left side of the first rectangular image with respect to the reference line. Then, the coordinates of the upper left corner vertex of each rectangular frame in the first rectangular frame set are obtained, a second statistical value of the coordinates of these upper left corner vertices is calculated, and the coordinates of the first reference point are obtained. Alternatively, the coordinates of the lower left corner vertex of each rectangular frame in the first rectangular set are obtained, a second statistical value of the coordinates of these lower left corner vertices is calculated, and the coordinates of the first reference point are obtained. Alternatively, the coordinates of the upper left corner vertex and the lower left corner vertex of each rectangular frame in the first rectangular set are obtained, a second statistical value of the coordinates of these upper left corner vertices and the lower left corner vertex is calculated, and the coordinates of the first reference point are obtained. Furthermore, a straight line that passes through the first reference point and has a tilt angle of the first target tilt angle is generated, and is used as the first outer frame line of the frameless table.

[0050] A second target inclination angle is determined based on a first statistical value of an angle of the right frame side of each rectangular frame in the second rectangular frame set corresponding to the right side of the first rectangular image with respect to a reference line. Then, the coordinates of the upper right corner vertex of each rectangular frame in the second rectangular frame set are obtained, second statistical values ​​of these upper right corner vertices are calculated, and the coordinates of the second reference point are obtained. Alternatively, the coordinates of the lower right corner vertex of each rectangular frame in the second rectangular set are obtained, second statistical values ​​of these lower right corner vertices are calculated, and the coordinates of the second reference point are obtained. Alternatively, the coordinates of the upper right corner vertex and the lower right corner vertex of each rectangular frame in the second rectangular set are obtained, second statistical values ​​of the coordinates of these upper right corner vertices and the lower right corner vertices are calculated, and the coordinates of the second reference point are obtained. Furthermore, a straight line that passes through the second reference point and has an inclination angle of the second target inclination angle is generated, and the straight line is used as the second outer frame line of the frameless table.

[0051] A third target tilt angle is determined based on a first statistical value of an angle of the upper frame side of each rectangular frame in the third rectangular frame set corresponding to the top side of the first rectangular image with respect to the reference line. Then, the coordinates of the upper left corner vertex of each rectangular frame in the third rectangular frame set are obtained, a second statistical value of these upper left corner vertices is calculated, and the coordinates of the third reference point are obtained. Alternatively, the coordinates of the upper right corner vertex of each rectangular frame in the third rectangular set are obtained, a second statistical value of these upper right corner vertices is calculated, and the coordinates of the third reference point are obtained. Alternatively, the coordinates of the upper left corner vertex and the upper right corner vertex of each rectangular frame in the third rectangular set are obtained, a second statistical value of the coordinates of these upper left corner vertices and the upper right corner vertices is calculated, and the coordinates of the third reference point are obtained. Furthermore, a straight line that passes through the third reference point and has a tilt angle of the third target tilt angle is generated, and the straight line is used as the third outer frame line of the frameless table.

[0052] A fourth target inclination angle is determined based on a first statistical value of an angle of the bottom frame side of each rectangular frame in the fourth rectangular frame set corresponding to the bottom side of the first rectangular image with respect to the reference line. Then, the coordinates of the lower left corner vertex of each rectangular frame in the fourth rectangular frame set are obtained, a second statistical value of the lower left corner vertex is calculated, and the coordinates of the fourth reference point are obtained. Alternatively, the coordinates of the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained, a second statistical value of the lower right corner vertex is calculated, and the coordinates of the fourth reference point are obtained. Alternatively, the coordinates of the lower left corner vertex and the lower right corner vertex of each rectangular frame in the fourth rectangular set are obtained, a second statistical value of the coordinates of the lower left corner vertex and the lower right corner vertex are calculated, and the coordinates of the fourth reference point are obtained. Furthermore, a straight line that passes through the fourth reference point and has an inclination angle of the fourth target inclination angle is generated, and the straight line is set as the fourth outer frame line of the frameless table.

[0053] In step 14, four intersection points are determined based on the four outer borders of the frameless table.

[0054] Here, based on the four outer borders of the frameless table obtained in step 13, the intersections of these outer borders, that is, four intersection points, are obtained.

[0055] In step 15, a perspective transformation is performed based on the four intersection points and the four predetermined target points to form a perspective transformation matrix.

[0056] Here, the four predetermined target points are the four vertices of the perspective-transformed frameless table, which respectively correspond to the four intersection points obtained in step 14. According to the coordinates of the intersection points and the target points, a perspective transformation matrix is ​​formed through conventional processing methods.

[0057] Specifically, one of the four intersections is set as a predetermined target point, and the coordinates of the remaining predetermined target points are determined based on the coordinates of the arbitrary intersection and the distance between the intersection and the neighboring intersection. Hereinafter, an example will be described in which the lower left intersection is set as the arbitrary intersection.

[0058] The coordinates of the lower left intersection point where the first outer frame line and the fourth outer frame line intersect are (x1, y1).

[0059] The coordinates of the upper left intersection point where the first outer frame line and the third outer frame line intersect are (x2, y2).

[0060] The coordinates of the intersection point at the bottom right where the second outer frame line and the fourth outer frame line intersect are (x3, y3).

[0061] The coordinates of the upper right intersection point where the second outer frame line and the third outer frame line intersect are (x4, y4).

[0062] If the frameless table does not need to be resized, select the lower left intersection as the first predetermined target point. That is, the coordinates of the first predetermined target point corresponding to the lower left intersection are (x1, y1). Then, calculate the first distance L1 between the lower left intersection and the upper left intersection, and generate the coordinates (x1, y1+L1) of the second predetermined target point corresponding to the upper left intersection. Also, calculate the second distance L2 between the lower left intersection and the lower right intersection, and generate the coordinates (x1+L2, y1) of the third predetermined target point corresponding to the lower right intersection. Finally, obtain the coordinates (x1+L2, y1+L1) of the fourth predetermined target point corresponding to the upper right intersection.

[0063] If the frameless table does not need to be resized, the size is resized for the first distance and the second distance according to a preset zoom ratio α, and the coordinates of the four predetermined target points are as follows: the first predetermined target point corresponding to the lower left intersection is (x1, y1), the second predetermined target point corresponding to the upper left intersection is (x1, y1+α*L1), the third predetermined target point corresponding to the lower right intersection is (x1+α*L2, y1), and the fourth predetermined target point corresponding to the upper right intersection is (x1+α*L2, y1+α*L1).

[0064] In step 16, the first rectangular image is corrected based on the perspective transformation matrix to generate a corrected second rectangular image.

[0065] Here, the first rectangular image is subjected to tilt correction using a perspective transformation matrix to obtain a corrected second rectangular image.

[0066] According to the above steps, the embodiment of the present invention determines each outer border of the frameless table based on the skew angle and vertex coordinates of the rectangular frame obtained by text detection. Since the skew angle and vertex coordinates of multiple rectangular frames are comprehensively considered, the accuracy of the feature points of the obtained frameless table image can be improved, and the performance of the subsequent image skew correction process can be improved.

[0067] In addition, the embodiment of the present invention provides a skew correction device for implementing the above skew correction method. As shown in Fig. 6, the skew correction device for a frameless table according to the embodiment of the present invention includes an image acquisition module 601 for acquiring a first rectangular image of a frameless table including text, a text detection module 602 for performing text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion, a rectangular frame set including at least one rectangular frame near each side of the first rectangular image, a target skew angle is determined based on the skew angle of a target frame side that is the frame side closest to the side in a rectangular frame in the rectangular frame set, and a text detection module 603 for detecting the text portion in the first rectangular image. the frame line generating module 603 for determining a reference point based on a target vertex, which is a vertex through which the target frame side passes; the frame line generating module 603 for generating a straight line that passes through the reference point and has a slant angle of the target slant angle as the frame line of the frameless table; the intersection generating module 604 for determining four intersections based on four frame lines of the frameless table; the matrix obtaining module 605 for performing perspective transformation based on the four intersections and four predetermined target points to form a perspective transformation matrix; and the image correction module 606 for performing correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image.

[0068] Through the above modules, the embodiment of the present invention can improve the accuracy of the feature points of the captured frameless table image, and improve the performance of the subsequent image skew correction process.

[0069] Preferably, the outer frame line generation module 603 includes a first identification module that identifies a matrix to which a rectangular frame in the first rectangular image belongs, and among the rectangular frames in the same row, the distance between the upper frame extension lines of any two rectangular frames is smaller than a first threshold, the distance between the lower frame extension lines of any two rectangular frames is smaller than a second threshold, and the distance between the left frame extension lines of any two rectangular frames in the same column is smaller than a third threshold; a first selection module that selects a rectangular frame in the first column for the left side of the first rectangular image and constitutes a first rectangular frame set; a second selection module that selects a rectangular frame in the last column for the right side of the first rectangular image and constitutes a second rectangular frame set; a third selection module that selects a rectangular frame in the first row for the top side of the first rectangular image and constitutes a third rectangular frame set; and a fourth selection module that selects a rectangular frame in the last row for the bottom side of the first rectangular image and constitutes a fourth rectangular frame set.

[0070] Preferably, the outer border generation module 603 includes a first calculation module that calculates the angle of a target frame of each rectangular frame in the rectangular frame set relative to a reference line, which is a predetermined horizontal or vertical line, and a second calculation module that calculates a first statistical value, which is the median or average value of the angles of the target frame of each rectangular frame relative to the reference line, as the target inclination angle.

[0071] Preferably, the outer border generation module 603 further includes a first acquisition module for acquiring coordinates of target vertices of each rectangular frame in the rectangular frame set, and a third calculation module for calculating a second statistical value, which is the median or average value of the acquired coordinates of the target vertices, as the coordinate of the reference point.

[0072] Preferably, the first acquisition module further includes acquiring the coordinates of the upper left and / or lower left corner vertex of each rectangular frame in the first rectangular set with respect to the left side of the first rectangular image, acquiring the coordinates of the upper right and / or lower right corner vertex of each rectangular frame in the second rectangular set with respect to the right side of the first rectangular image, acquiring the coordinates of the upper left and / or upper right corner vertex of each rectangular frame in the third rectangular set with respect to the top side of the first rectangular image, and acquiring the coordinates of the lower left and / or lower right corner vertex of each rectangular frame in the fourth rectangular set with respect to the bottom side of the first rectangular image.

[0073] It should be noted that the device / system provided in the above embodiment is a frameless table tilt correction device corresponding to the frameless table tilt correction method. The implementation manner of the above embodiment can also be applied to the device embodiment, and achieves the same technical effects. Therefore, the tilt correction device of the embodiment of the present invention implements all the method steps implemented in the tilt correction method embodiment and achieves the same effects, and the detailed description of the parts similar to those of the method embodiment and the beneficial effects will be omitted.

[0074] 7 is a block diagram showing a hardware structure of a frameless table skew correction device according to an embodiment of the present invention. As shown in FIG. 7, the frameless table skew correction device 700 includes a processor 702 and a memory 704 for storing computer program instructions. When the computer program instructions are executed by the processor, the processor 702 obtains a first rectangular image of a frameless table including text; performs text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; generates a rectangular frame set including at least one rectangular frame near each side of the first rectangular image; and detects the text portion in the rectangular frame set by detecting the text portion near each side of the first rectangular image. The steps of determining a target inclination angle based on the inclination angle of the target frame side which is the closest frame side, determining a reference point based on a target vertex which is a vertex through which the target frame side passes in a rectangular frame in the rectangular frame set, and generating a straight line which passes through the reference point and has an inclination angle of the target inclination angle as an outer frame line of the frameless table; determining four intersection points based on four outer frame lines of the frameless table; performing perspective transformation based on the four intersection points and four specified target points to form a perspective transformation matrix; and performing correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image are realized.

[0075] It should be noted that the device / system provided in the above embodiment is a frameless table tilt correction device corresponding to the frameless table tilt correction method. The implementation manner of the above embodiment can also be applied to the device embodiment, and achieves the same technical effects. Therefore, the tilt correction device of the embodiment of the present invention implements all the method steps implemented in the tilt correction method embodiment and achieves the same effects, and the detailed description of the parts similar to those of the method embodiment and the beneficial effects will be omitted.

[0076] Furthermore, as shown in FIG. 7, the frameless table tilt correction device 700 includes a network interface 701 , an input device 703 , a hard disk 705 , and a display device 706 . These interfaces and devices are connected to each other via a bus architecture. The bus architecture may include any number of interconnected buses and bridges. Specifically, various circuits such as one or more central processing units (CPUs) and / or graphics processors (GPUs) represented by processor 702, and one or more memories represented by memory 704 are connected. The bus architecture may also connect various other circuits such as peripherals, voltage regulators, power management circuits, etc. The bus architecture allows communication between these components. In addition to a data bus, the bus architecture may include a power bus, a control bus, and a status signal bus. All of these are well known in the art and will not be described in detail. The network interface 701 is connected to a network (eg, the Internet, a local area network, etc.), receives data such as original training samples from the network, and stores the received data in the hard disk 705 .

[0077] The input device 703 receives various commands input by an operator and transmits them to the processor 702 for execution. The input device 703 includes a keyboard or a pointing device (e.g., a mouse, a trackball, a touch board, or a touch screen, etc.).

[0078] Display device 706 displays the results of processor 702 executing the instructions, such as displaying model training progress.

[0079] The memory 704 stores programs and data necessary for running an operating system (OS), as well as data such as intermediate results during calculations by the processor 702 .

[0080] It should be noted that memory 704 in embodiments of the invention may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. Wherein non-volatile memory may be read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM) or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. Memory 704 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0081] In some embodiments, memory 704 stores an operating system 7041 and application programs 7042, which may be executable modules or data structures or submodules or extensions thereof.

[0082] Here, the operating system 7041 includes various system programs, such as a framework layer, a core library layer, and a drive layer, and is used to realize various core operations and hardware-based tasks. The application programs 7042 include various application programs, such as a web browser, and are used to realize various application operations. The program for executing the method according to this embodiment is included in the application programs 7042.

[0083] The method according to the embodiment of the present invention is applied to or implemented by the processor 702. The processor 702 is an integrated circuit board capable of processing signals. Each step of the method is implemented by an integrated logic circuit in hardware or instructions in software form in the processor 702. The processor 702 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-flash programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing each method, step, and logic box disclosed in the embodiment of the present invention. The general-purpose processor may be a microprocessor or any general processor. Each step of the method according to the embodiment of the present invention may be implemented by being executed by a decoder in hardware, or may be implemented by a combination of hardware and software that can be implemented in the decoder. The software module is stored in a storage medium mature in the field, such as a random memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The processor 702 reads information from the memory 704, which comprises a storage medium on which the software is stored, and configures the hardware to implement the steps of the above-mentioned method.

[0084] The above-described embodiments may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof, whereby, for hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units performing the functions of the present invention, or a combination thereof.

[0085] Regarding software implementation, the above technology is realized by modules (e.g., processes, functions, etc.) that realize the functions described above. The software code is stored in a memory and executed by a processor. The memory may be implemented inside or outside the processor.

[0086] Specifically, when the computer program is executed by the processor 702, the following steps are realized: identifying a matrix to which a rectangular frame in the first rectangular image belongs, and among the rectangular frames in the same row, the distance between the upper frame extension lines of any two rectangular frames is smaller than a first threshold, the distance between the lower frame extension lines of any two rectangular frames in the same column is smaller than a third threshold; selecting a rectangular frame in the first column for the left side of the first rectangular image to construct a first rectangular frame set, selecting a rectangular frame in the last column for the right side of the first rectangular image to construct a second rectangular frame set, selecting a rectangular frame in the first row for the top side of the first rectangular image to construct a third rectangular frame set, and selecting a rectangular frame in the last row for the bottom side of the first rectangular image to construct a fourth rectangular frame set.

[0087] In particular, when the computer program is executed by the processor 702, the steps of calculating the angle of the target frame of each rectangular frame in the set of rectangular frames relative to a reference line, which is a predetermined horizontal or vertical line, and calculating a first statistical value, which is the median or average value of the angles of the target frames of each rectangular frame relative to the reference line, as the target inclination angle, are realized.

[0088] In particular, when the computer program is executed by the processor 702, the steps of obtaining the coordinates of the target vertices of each rectangular frame in the set of rectangular frames; and calculating a second statistical value, which is the median or average value of the coordinates of the obtained target vertices, as the coordinate of the reference point, are realized.

[0089] In particular, when the computer program is executed by the processor 702, the following steps are realized: acquiring the coordinates of the upper left and / or lower left vertex of each rectangular frame in the first rectangular collection for the left side of the first rectangular image; acquiring the coordinates of the upper right and / or lower right vertex of each rectangular frame in the second rectangular collection for the right side of the first rectangular image; acquiring the coordinates of the upper left and / or upper right vertex of each rectangular frame in the third rectangular collection for the top side of the first rectangular image; and acquiring the coordinates of the lower left and / or lower right vertex of each rectangular frame in the fourth rectangular collection for the bottom side of the first rectangular image.

[0090] The frameless table tilt correction device provided in the above embodiment realizes all the method steps realized in the frameless table tilt correction method and has the same effects, so detailed descriptions of the parts similar to those in the method embodiment and the beneficial effects will be omitted.

[0091] Furthermore, in some embodiments of the present invention, a computer-readable storage medium is provided on which a program is stored, and the program is executed by a processor to realize the steps of obtaining a first rectangular image of a frame-less table including text; performing text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; generating a rectangular frame set including at least one rectangular frame near each side of the first rectangular image, and determining a target inclination angle based on the inclination angle of a target frame side that is the frame side closest to the side in a rectangular frame in the rectangular frame set; determining a reference point based on a target vertex that is a vertex through which the target frame side passes in a rectangular frame in the rectangular frame set, and generating a straight line that passes through the reference point and has a inclination angle of the target inclination angle as an outer frame line of the frame-less table; determining four intersection points based on four outer frame lines of the frame-less table; performing perspective transformation based on the four intersection points and four predetermined target points to form a perspective transformation matrix; and performing correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image.

[0092] By executing the program on a processor, all aspects of the frameless table tilt correction method are realized and the same technical effects are achieved, so in order to avoid duplication, a description thereof will be omitted here.

[0093] Those skilled in the art of the present invention can easily imagine that the units and algorithm steps of each example described in the above disclosed embodiments can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by either hardware or software depends on the specific application and design constraints of the invention. Those skilled in the art can realize the above functions in a way that suits the specific application, but it should not go beyond the scope of the present invention.

[0094] In addition, for convenience and conciseness of explanation, detailed descriptions of the specific operating processes of the above systems, devices and units will be omitted, since it is clear to those skilled in the art that reference can be made to the corresponding processes in the above embodiments.

[0095] It is easily conceivable that the method and apparatus disclosed in the multiple embodiments of the present invention can be realized in other forms. For example, the above-described apparatus is merely schematic. For example, the division of the units is merely one example of logical function allocation, and a different division method may be adopted when actually realizing the present invention. For example, a plurality of units or modules may be combined or integrated into another system, or some functions may be omitted or not executed. Note that the mutual connections, direct connections, or communicative connections shown or disclosed above are connections via interfaces. Indirect connections or communicative connections between devices or units may be electrical, mechanical, or other forms of connections.

[0096] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, i.e., they may be in the same location or distributed across multiple network units. Depending on actual needs, some or all of the units may be selected to achieve the objectives of the embodiments of the present invention.

[0097] Note that each functional unit according to the embodiment of the present invention may be integrated into one processing unit, may be physically independent, or may be integrated into one unit consisting of two or more functional units.

[0098] The functions can be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. In this case, the technical solution of the present invention is essentially or contributes to the prior art or the technical solution is expressed in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute all or part of the steps of the method according to each embodiment of the present invention. The storage medium includes various media capable of storing program code, such as a USB memory, a removable disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0099] The above description is a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, and any modifications or replacements that can be easily conceived by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of the present invention should be based on the claims.

Claims

1. A method for correcting inclination of a frameless table, the method being performed by a frameless table inclination correction device, comprising the steps of: obtaining a first rectangular image of a frameless table containing text; performing text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; a step of generating, for each side of the first rectangular image, a rectangular frame set including at least one rectangular frame in the vicinity of the side, determining a target inclination angle based on the inclination angle of a target frame side of a rectangular frame in the rectangular frame set, determining a reference point based on a target vertex of a rectangular frame in the rectangular frame set, and generating a straight line that passes through the reference point and has a inclination angle of the target inclination angle as an outer frame line of the frameless table, wherein the target frame side is a frame side in the rectangular frame that is closest to the side, and the target vertex is a vertex through which the target frame side passes; determining four intersection points based on four outer border lines of the frameless table; performing a perspective transformation based on the four intersection points and four predetermined target points to obtain a perspective transformation matrix; and performing a correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image. A method for correcting tilt of a frameless table.

2. The step of generating a rectangular frame set for each side of the first rectangular image includes: a step of identifying a matrix to which a rectangular frame in the first rectangular image belongs, in which, in rectangular frames in the same row, a distance between upper frame extension lines of any two rectangular frames is smaller than a first threshold value, and a distance between lower frame extension lines of any two rectangular frames is smaller than a second threshold value, and, in rectangular frames in the same column, a distance between left frame extension lines of any two rectangular frames is smaller than a third threshold value; selecting a rectangular frame in a first column for a left edge of the first rectangular image and constructing a first rectangular frame set, selecting a rectangular frame in a last column for a right edge of the first rectangular image and constructing a second rectangular frame set, selecting a rectangular frame in a first row for a top edge of the first rectangular image and constructing a third rectangular frame set, and selecting a rectangular frame in a last row for a bottom edge of the first rectangular image and constructing a fourth rectangular frame set.

2. The method for correcting tilt of a frameless table according to claim 1.

3. The step of determining a target inclination angle based on an inclination angle of a target frame side of a rectangular frame in the rectangular frame set includes: A step of calculating an angle of a target frame side of each rectangular frame in the rectangular frame set with respect to a reference line, the reference line being a predetermined horizontal line or vertical line; calculating a first statistical value of an angle of each rectangular frame with respect to a reference line of the target frame as the target tilt angle, the first statistical value being a median or average value of the angles.

3. The method for correcting tilt of a frameless table according to claim 2.

4. The step of determining a reference point based on a target vertex of a rectangular frame in the rectangular frame set includes: obtaining coordinates of a target vertex of each rectangular frame in the set of rectangular frames; calculating a second statistical value of the coordinates of the acquired target vertices as the coordinates of the reference point, the second statistical value being a median or an average value of the coordinates of the target vertices; 3. The method for correcting tilt of a frameless table according to claim 2.

5. The step of obtaining coordinates of a target vertex of each rectangular frame in the set of rectangular frames includes: obtaining the coordinates of the top left vertex and / or the bottom left vertex of each rectangular frame in the first set of rectangles relative to the left side of the first rectangular image; obtaining the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second set of rectangles relative to the right side of the first rectangular image; obtaining the coordinates of the top left corner vertex and / or the top right corner vertex of each rectangular frame in the third set of rectangles relative to a top edge of the first rectangular image; and obtaining the coordinates of the bottom left vertex and / or bottom right vertex of each rectangular frame in the fourth set of rectangles relative to a bottom edge of the first rectangular image.

5. The method for correcting tilt of a frameless table according to claim 4.

6. an image capture module for capturing a first rectangular image of a frameless table including text; a text detection module that performs text detection on the first rectangular image to obtain a text portion in the first rectangular image and a rectangular frame circumscribing the text portion; an outer frame line generation module that generates, for each side of the first rectangular image, a rectangular frame set including at least one rectangular frame in the vicinity of the side, determines a target inclination angle based on an inclination angle of a target frame side of a rectangular frame in the rectangular frame set, determines a reference point based on a target vertex of a rectangular frame in the rectangular frame set, and generates a straight line that passes through the reference point and has an inclination angle of the target inclination angle as an outer frame line of the frameless table, wherein the target frame side is a frame side of the rectangular frame that is closest to the side, and the target vertex is a vertex through which the target frame side passes; an intersection generation module that determines four intersections based on the four outer frame lines of the frameless table; a matrix acquisition module for performing a perspective transformation based on the four intersections and four predetermined target points to acquire a perspective transformation matrix; an image correction module that performs correction on the first rectangular image based on the perspective transformation matrix to generate a corrected second rectangular image. A frameless table tilt correction device.

7. The outer frame line generation module includes: a first identification module that identifies a matrix to which a rectangular frame in the first rectangular image belongs, where the distance between the upper frame extension lines of any two rectangular frames in the same row is smaller than a first threshold value, and the distance between the lower frame extension lines of any two rectangular frames in the same column is smaller than a third threshold value; a first selection module for selecting rectangular boxes in a first row relative to a left edge of the first rectangular image to form a first set of rectangular boxes; a second selection module for selecting a last row of rectangular frames for a right edge of the first rectangular image to form a second set of rectangular frames; a third selection module for selecting rectangular boxes in a first row relative to a top edge of the first rectangular image to form a third set of rectangular boxes; a fourth selection module for selecting a rectangular frame in the last row relative to a bottom edge of the first rectangular image to form a fourth set of rectangular frames.

7. The tilt correction device for a frameless table according to claim 6.

8. The outer frame line generation module includes: A first calculation module that calculates an angle of a target frame side of each rectangular frame in the rectangular frame set with respect to a reference line, the reference line being a predetermined horizontal line or vertical line; a second calculation module that calculates a first statistical value of an angle of each rectangular frame with respect to a reference line of the target frame as the target tilt angle, the first statistical value being a median or average value of the angles.

8. The frameless table tilt correction device according to claim 7.

9. The outer frame line generation module includes: a first acquisition module for acquiring coordinates of a target vertex of each rectangular frame in the set of rectangular frames; and a third calculation module for calculating a second statistical value of the coordinates of the acquired target vertices as the coordinates of the reference points, the second statistical value being a median or an average value of the coordinates of the target vertices.

8. The frameless table tilt correction device according to claim 7.

10. The first acquisition module is obtain the coordinates of the upper left corner vertex and / or the lower left corner vertex of each rectangular frame in the first set of rectangles with respect to the left side of the first rectangular image; obtain the coordinates of the upper right corner vertex and / or the lower right corner vertex of each rectangular frame in the second set of rectangles with respect to the right side of the first rectangular image; obtain the coordinates of the top left corner vertex and / or the top right corner vertex of each rectangular frame in the third set of rectangles relative to a vertex of the first rectangular image; obtaining the coordinates of the bottom left corner vertex and / or the bottom right corner vertex of each rectangular frame in the fourth set of rectangles relative to a bottom side of the first rectangular image; 10. The frameless table tilt correction device according to claim 9.

11. A program that causes a processor to execute the method for correcting the inclination of a frameless table according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Method and device for correcting inclined text image, medium and electronic equipment

    CN110427939A

  • Medical copy text tilt correction method, electronic equipment and readable storage medium

    CN113177899A

  • Image identification method and device, equipment and readable storage medium

    CN113762054A

  • Document labeling method and device, electronic equipment and storage medium

    CN114387598A

  • Image viewpoint conversion device and method

    JP2018045691A