Page distortion correction method and system, electronic device and computer readable medium

Abstract

A page distortion correction method and system, an electronic device and a computer readable medium. The page distortion correction method comprises: determining a target text line to be corrected, and selecting a plurality of check points in the target text line, the plurality of check points satisfying that the connecting line of the plurality of check points is coincident with or parallel to a correction reference line when the page is not distorted; comparing the positions of the plurality of check points with the correction reference line, and calculating the offset amount of the check points to be corrected; and correcting each character of the target text line according to the calculated offset amount. The method is simple and efficient, has good restoration, strong adaptability, and the correction result is more stable.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition and processing, and more particularly relates to a page distortion correction method and system, an electronic device, and a computer readable medium. BACKGROUND

[0002] With the progress of technology, students can now take pictures of difficult problems they cannot solve using their mobile phones and search for answers on the Internet, greatly facilitating the search for knowledge and making up for the lack of teachers. During the process of taking pictures with a mobile phone, the paper carrying the problem may be bent and deformed, causing the page captured to be stretched and deformed, increasing the difficulty of text recognition. In the field of traditional scanners and copiers, there is also a simple correction process for distorted pages, but it only identifies the four corners of the page for simple coefficient calculation and cannot correct the font distortion caused by page curling. Therefore, the existing page distortion correction technology still has great deficiencies, and there is an urgent need to develop a new correction method and system. SUMMARY

[0003] Therefore, the main purpose of the present application is to provide a page distortion correction method and system, an electronic device, and a computer readable medium, in order to at least partially solve at least one of the above technical problems.

[0004] In order to achieve the above-mentioned purpose, as a first aspect of the present application, a page distortion correction method is provided, comprising the following steps:

[0005] Determine the target text line to be corrected, and select a plurality of check points in the target text line, wherein the plurality of check points satisfy that the connecting line of the plurality of check points coincides with or is parallel to the correction reference line when the page is not distorted;

[0006] Compare the positions of the plurality of check points with the correction reference line, and calculate the offset amount of the check points that need to be corrected;

[0007] Correct each character of the target text line according to the calculated offset amount.

[0008] Optionally, before determining the target text line to be corrected, the method further comprises: identifying the text line in the page image to be corrected.

[0009] Optionally, the step of identifying the text line in the page image to be corrected comprises any one or a combination of multiple ways:

[0010] Detect the text target in the page image to be corrected, and obtain the position information of the text target; based on the position information of the text target, determine the text line located on the same line; or,

[0011] identify the text line in the page image to be corrected according to the image gray scale of the page image to be corrected, if there is a gray scale area extending horizontally or vertically and the width meets a preset value, the text line is determined; or

[0012] The text line is detected by an artificial intelligence model for detecting a text line in an image.

[0013] Optionally, the step of detecting a text target in the page image to be corrected and obtaining position information of the text target comprises any one or a combination of the following modes:

[0014] The text target is detected by performing OCR recognition on the page image to be corrected, and position information of the detected text target is obtained.

[0015] The text target is detected by an artificial intelligence model for detecting a text target in an image, and position information of the detected text target is obtained.

[0016] Optionally, the step of selecting a plurality of verification points in the target text line comprises: based on the length and font size of the target text line, setting a proportion at equal intervals in the font height to sample a plurality of verification points.

[0017] Optionally, a point at a set proportion position in the font width and / or height direction of the text is selected as a verification point.

[0018] Optionally, the text target is a printed font.

[0019] Optionally, the method further comprises: correcting the text of the adjacent line of the target text line according to the calculated offset.

[0020] Optionally, the method further comprises: correcting the area consistent with the distortion of the target text line according to the calculated offset.

[0021] Optionally, each target text line to be corrected corresponds to a correction reference line, and a plurality of correction reference lines on the same page to be corrected are parallel.

[0022] Optionally, the method further comprises: performing the above operation on all target text lines in the page to be corrected, so as to correct the entire page to be corrected by taking the selected verification points in the target text line as anchor points.

[0023] Optionally, the correction reference line is a horizontal line corresponding to the target text line.

[0024] In the step of comparing the positions of the plurality of check points with the correction reference line and calculating the offset of the check points that need to be corrected, the leftmost or rightmost point is taken as the reference point, and the relative offset in the longitudinal direction is calculated for all the check points relative to the reference point;

[0025] Optionally, the step of calculating the offset of the check points that need to be corrected further comprises: comparing the positions of the plurality of check points with the theoretical position of each character calculated according to the width and the number of characters of the target character line, to obtain the relative offset of all the check points in the transverse direction;

[0026] Optionally, during the correction, for the longitudinal direction, the font of the reference point is not moved, and the fonts of the other points are moved to a line that passes through the reference point and is parallel to the correction reference line;

[0027] Optionally, during the correction, for the transverse direction, the font of the reference point is not moved, and the fonts of the other points are moved according to the relative offset;

[0028] Preferably, before the step of calculating the offset of the check points that need to be corrected, the distortion degree of the page to be corrected is evaluated;

[0029] Further preferably, the step of evaluating the distortion degree of the page to be corrected is realized by comparing the offset with the width and height of the font in the target character line, respectively;

[0030] Further preferably, when the evaluation score of the distortion degree of the page to be corrected is lower than a first threshold value, the correction procedure is ended;

[0031] Optionally, in the step of calculating the offset of the check points that need to be corrected, the offset of the check auxiliary points on the characters between two selected check points is calculated by interpolation or curve fitting; and subsequently, the characters between the two selected check points are corrected according to the offset of the check auxiliary points.

[0032] Optionally, before the step of determining the target character line to be corrected, a pre-correction step of the page is further included;

[0033] Optionally, the step of pre-correction of the page comprises: determining the top corners of the paper page from the image of the page to be corrected; comparing the coordinates of the top corners with the corresponding reference horizontal line and / or reference vertical line, calculating the offset of each top corner that needs to be corrected; and pre-correcting the image of the page to be corrected according to the calculated offset;

[0034] Optionally, the top corners in the image of the page to be corrected are automatically recognized by a trained artificial intelligence model, and the position information of the recognized top corners is stored in a database;

[0035] Optionally, the pre-correction of the page comprises: removing irrelevant information in the page image to be corrected, and stretching the page region in the page image to be corrected to a normalized shape corresponding to the page.

[0036] Optionally, the step of pre-correcting the page image according to the calculated relative offset further comprises:

[0037] The relative offset of the pre-correction auxiliary point between any two of the top corners is calculated by interpolation or curve fitting.

[0038] As a second aspect of the present application, a page distortion correction system is also provided, comprising:

[0039] A check point selection unit is configured to determine a target line of text to be corrected, and select a plurality of check points in the target line of text, the plurality of check points satisfying that a line connecting the plurality of check points is coincident with or parallel to a correction reference line when the page is not distorted.

[0040] An offset determination unit is configured to compare the positions of the plurality of check points with the correction reference line, and calculate the offset of the check points to be corrected.

[0041] A correction unit is configured to correct each character in the target line of text according to the calculated offset.

[0042] As a third aspect of the present application, an electronic device is also provided, comprising a processor and a memory, the memory being configured to store a computer executable program, when the computer executable program is executed by the processor, the processor executes the page distortion correction method as described above.

[0043] As a fourth aspect of the present application, a computer readable medium is also provided, storing a computer executable program, when the computer executable program is executed, the page distortion correction method as described above is implemented.

[0044] Based on the above technical solutions, the page distortion correction method and system of the present application have at least one of the following beneficial effects relative to the prior art:

[0045] The correction method of the present application is simple and efficient, and can quickly correct the distortion of various pages / characters caused by the deformation of the plane on which the pages / characters are carried, and has good restoration;

[0046] The correction method of the present application is highly adaptable, and is suitable for the situation of page distortion and deformation caused by the bending and wrinkling of the carrying paper;

[0047] Compared with other correction schemes, especially the scheme of directly generating a correction effect through a neural network model, the correction result given by the present application is more stable and is not prone to result deterioration cases.

[0048] The present application has an additional advantage that a quantitative value of the distortion degree of the current distorted page can be given, thereby assisting the upstream task in strategy scheduling design, for example, not performing correction when the distortion degree is low. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a block flowchart of the page distortion correction method of the present application;

[0050] Figure 2 is a structural schematic diagram of the page distortion correction system of the present application;

[0051] Figure 3 is a structural schematic diagram of the electronic device of the present application;

[0052] Figure 4 is a schematic diagram of the computer readable medium of the present application;

[0053] Figure 5 is a schematic diagram of calculating the check point coordinate value according to the line of text in the present application;

[0054] Figure 6 is an enlarged view of calculating the check point coordinate value according to the font;

[0055] Figure 7 、 8 is a comparative schematic photo of page pre-correction of the embodiment of the present application;

[0056] Figure 9 is a photo of actual processing effect of the embodiment of the present application. DETAILED DESCRIPTION

[0057] In the introduction of the specific embodiments, the detailed description of the structure, performance, effect or other characteristics is to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can implement the present application without the above-mentioned structure, performance, effect or other characteristics in specific cases.

[0058] The flowchart in the drawings is only an example of flow demonstration, does not represent that all the contents, operations and steps in the flowchart must be included in the scheme of the present application, nor does it represent that it must be executed in the order shown in the figure. For example, some operations / steps in the flowchart can be decomposed, some operations / steps can be combined or partially combined, etc. The execution order shown in the flowchart can be changed according to the actual situation without departing from the inventive concept of the present application.

[0059] The blocks in the drawings Figure 1 Generally represent functional entities and do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processing unit devices and / or microcontroller devices.

[0060] The same reference signs in different drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts can be omitted hereinafter. It should also be understood that although the first, second, third, etc. denotative adjectives can be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these adjectives. That is, these adjectives are only used to distinguish one from another. For example, a first device can also be referred to as a second device without departing from the essential technical solutions of the present application. In addition, the terms "and / or", "and / or" mean all combinations of one or more of the listed items.

[0061] The present application is proposed for the technical problem of how to correct the distortion of a page when taking a photo of the test content including printed characters and handwritten scripts on the paper, which can overcome the defects of the traditional method that can only detect the four corners of the page for simple stretching and is less applicable to the scene. The method of the present application selects part of anchor points (check points) on the characters as a reference to calibrate the characters, and the correction ability is more powerful, more flexible and efficient. The technical solutions of the present application will be described below in combination with the drawings.

[0062] As Figure 1 shown, the present application proposes a page distortion correction method, which specifically includes the following steps:

[0063] Step one, determine the target character line to be corrected, and select a plurality of check points in the target character line, the plurality of check points meet: the connecting line of the plurality of check points is coincident or parallel with the correction reference line when the page is not distorted;

[0064] Exemplarily, the correction reference line can be a parallel line of the edge line of the image, and after correction, the character line is parallel to the page edge line, which conforms to the reading habit of people.

[0065] Exemplarily, as Figure 5 shown, the correction reference line can be located at the middle position of the character line to be corrected.

[0066] There are various ways to select the check points, as long as the check points are on the same straight line when the page is not distorted. The correction reference line is generally coincident with or parallel to the straight line. Alternatively, the check points are generally equidistantly distributed.

[0067] On the basis of generally equidistant distribution, the check points are more densely distributed in the area where the distortion is more serious. As shown in Figure 9 The edge area of the image is relatively more seriously distorted compared with the middle area of the page, and more check points can be selected in the edge area of the image. The more check points, the more calculation, but the correction effect is better. This scheme can balance the correction effect and the calculation amount.

[0068] Step two, compare the positions of the check points with the correction reference line, and calculate the offset of the check points that need to be corrected;

[0069] This step can calculate the distance from the position of the check point to the correction reference line, and calculate the distance of pulling the check points to the same straight line, i.e. the offset of the check points that need to be corrected.

[0070] Step three, correct the positions of the characters of the target text line according to the calculated offset.

[0071] This step moves the characters at the check points as a whole according to the offset calculated in step three, so as to pull the characters of the target text line, i.e. arrange them on a straight line. The offset of the characters between two check points can be determined according to the offset of the two check points that need to be corrected, for example, the average value of the offset of the two check points that need to be corrected.

[0072] Further, for the same target text line but without check points, the offset can also be predicted or obtained by fitting according to the trend of the offset of the check points of the line, and the correction is performed according to the obtained offset. The target text line can refer to the characters arranged in a row or a column in the current page image, including the target text line arranged in a column, because the row and the column are only different in arrangement direction, and the essence is the characters arranged in a line.

[0073] Before the step of determining the target text line to be corrected, the method can further include: identifying the text line in the page image to be corrected; wherein the step of identifying the text line in the page image to be corrected includes any one or a combination of the following ways:

[0074] detecting the text target in the page image to be corrected and obtaining the position information of the text target; determining the text line located on the same row based on the position information of the text target; or,

[0075] identify the text line in the page image to be corrected according to the image grayscale of the page image to be corrected, and determine the text line if there is a grayscale region extending horizontally in the page image to be corrected and the width of the grayscale region meets a preset value.

[0076] The text line is detected by an artificial intelligence model for detecting a text line in an image.

[0077] For example, the text line in the page image to be corrected can be identified according to the image grayscale of the page image to be corrected, and the text line is determined if there is a grayscale region extending horizontally in the page image to be corrected and the width of the grayscale region meets a preset value.

[0078] It should be noted that if it can be determined with a high probability that the page image to be corrected has a text line, the text line in the page image to be corrected can be identified according to the image information such as grayscale of the page image to be corrected, and the text line is determined if there is a grayscale region extending horizontally in the page image to be corrected and the width of the grayscale region meets a preset value.

[0079] The grayscale region extending horizontally means that the grayscale of the grayscale region is greater than a preset grayscale threshold. The context in which it can be determined with a high probability that the page image to be corrected has a text line includes, for example, in a specific APP such as a photograph searching for a question application and the like, and for example, if the previous step is combined with other detection (for example, in some APP, it can be determined in advance that the photographed picture is a page or other irrelevant things), it can be determined with a high probability that the page image to be corrected has a text line. Alternatively, the step of detecting the text target in the page image to be corrected and obtaining the position information of the text target comprises any one or a combination of the following modes:

[0080] The text target is detected by identifying the distorted part in the page image to be corrected and determining whether the identified content contains a representative feature of text, and the position information of the corresponding text target is obtained when it is determined that the identified content contains a representative feature of text, and the representative feature of text includes strokes, radicals, characters and letters.

[0081] The text target is detected by performing OCR identification on the page image to be corrected, and the position information of the identified text target is obtained.

[0082] The text target is detected by an artificial intelligence model for detecting a text target in an image, and the position information of the detected text target is obtained.

[0083] The picture is subjected to single character detection by an artificial intelligence model to detect all single text targets in the image and obtain the corresponding position information.

[0084] Optionally, the step of selecting several check points in the target text line comprises: selecting points at positions of a set proportion based on the font size of the target text line.

[0085] For example, several check points can be sampled at positions of a set proportion of the font height.

[0086] Preferably, points at positions of a set proportion in the width and / or height direction of the text are selected as check points; for example, the center position, one-third high and one-half wide position, two-thirds high and two-thirds wide position, and so on of the width and height of the text. There can or can not be strokes at the position, which does not affect the realization of the final purpose as long as the coordinate value can be obtained. For another example, the lowest point of the text can be selected as a check point.

[0087] The specific coordinate calculation process can adopt a known mathematical method. Since Chinese characters are square characters, even if there is stretching, twisting and deformation, they can still be regarded as roughly parallelogram or trapezoid, and the points at the specific proportion positions can be approximately calculated by proportion. Since Chinese characters account for a small proportion of the entire page, even if there is an error, the final effect is less affected.

[0088] Preferably, the target text is a printed font, and the correction effect is better. Because the printed font output is stable, unlike the handwritten font which can have defects such as font size being large or small and font row and column tilting in a certain direction when the page is not distorted.

[0089] Preferably, the check scheme is calculated for the text line of the printed font, and the handwritten text line near the text line of the printed font is corrected by using substantially the same scheme. Specifically, for the handwritten text line near the text line of the printed font, points with the same horizontal coordinate as a check point are corrected by using a substantially equal offset of the check point.

[0090] Optionally, points with the same horizontal coordinate (substantially located on the same vertical line on the image to be corrected) are corrected by using a substantially equal offset.

[0091] Optionally, the method further comprises: correcting the text of the adjacent line of the target text line according to the calculated offset.

[0092] Optionally, the method further comprises: correcting the area with the same distortion as the target text line according to the calculated offset. For example, the areas substantially located on the same vertical line on the image to be corrected have similar distortion effects, and the correction schemes (i.e. offsets) can be mutually referenced.

[0093] Optionally, each of the target text lines to be corrected corresponds to one of the correction reference lines, and a plurality of the correction reference lines on the same page to be corrected are arranged in parallel and at equal intervals; in this way, the distances between the corrected text lines are substantially equal.

[0094] Optionally, the method further comprises: performing the above operation on all target text lines in the page to be corrected, so as to correct the entire page to be corrected by taking the selected check points on the target text lines as anchor points.

[0095] As shown in Figure 5 the correction reference line can be a horizontal line corresponding to the target text line, and for a target text column, it is a vertical line. When correcting a column, it is necessary to determine that the selected check points should actually be located in the same column (i.e. actually in the same column when the page is not distorted). For example, the question numbers in the same question number level can be selected to ensure that they are actually located in the same column when the page is not distorted.

[0096] In the step of comparing the positions of the check points with the correction reference line and calculating the offset amount of the check points to be corrected, the leftmost or rightmost point is taken as a reference point, and the relative offset amount in the longitudinal direction of all check points is calculated with respect to the reference point; the offset amount of the reference point is 0, and the relative offset amount in the longitudinal direction of other check points is calculated with respect to the reference point. As shown in Figure 5 the leftmost check point is taken as the reference point, and the distance of each check point to the dashed line (correction reference line) is the relative offset amount in the longitudinal direction calculated with respect to the reference point.

[0097] Further, in other embodiments, the step of calculating the offset amount of the check points to be corrected can further comprise: comparing the positions of the check points with the theoretical position of each text calculated according to the width of the target text line and the number of text lines, to obtain the relative offset amount of all check points in the transverse direction.

[0098] In the present application, the vertical offset is the distance of the vertical coordinate value of the font to which the other point belongs deviating from the correction reference line of the reference point; and the horizontal offset can be estimated according to the width of the target text line and the number of characters in the line, for example. The following is an example. In the image of the page to be corrected, assuming that the width of the page is 1000 pixels, the normal width of the target text line should be 840 pixels, and due to the distortion of the target text line, the distance from the leftmost end of the leftmost character to the rightmost end of the rightmost character is reduced to 640 pixels, and the target text line contains 20 characters. Since the corrected characters should be arranged in equal width and equal character spacing (the advantage of square characters), 20 characters, 19 spaces, approximately 20 characters and 20 spaces, the distance of the first character from the reference point of the leftmost character can be approximately calculated according to the corresponding proportion of the width of the standard text line. The horizontal offset is the relative offset of the actual horizontal coordinate value of the check point relative to this distance.

[0099] In the correction, the reference point belongs to the font that does not move, and in the vertical direction, the font to which the other point belongs is moved to a line that passes through the reference point and is parallel to the correction reference line (such as the dashed line in Figure 5 In the horizontal direction, the font to which the other point belongs is moved relative to the reference point according to the horizontal offset.

[0100] As a preferred, before the step of calculating the offset of the check point that needs to be corrected, the distortion degree of the page to be corrected is evaluated.

[0101] Further preferably, the step of evaluating the distortion degree of the page to be corrected is realized by comparing the offset with the width and height of the font in the target text line, for example. Other lengths can also be used as references, such as the width and height of the entire target text line. The reference object is different, and the evaluation score is different.

[0102] Further preferably, when the evaluation score of the distortion degree of the page to be corrected is lower than the first threshold value, such as 5%, 8% or 10%, the correction program is ended. The specific value of the first threshold value can be determined according to the task requirements or experience value.

[0103] Optionally, in the step of calculating the offset of the check point that needs to be corrected, the offset of the check auxiliary point on the text between the two selected check points is calculated by interpolation or curve fitting; and the text between the two selected check points is subsequently corrected according to the offset of the check auxiliary point.

[0104] In the step of determining the target text line to be corrected, the step of pre-correcting the page is further included.

[0105] Optionally, the step of pre-correcting the page includes: determining the top corners of the paper page from the page image to be corrected; comparing the coordinates of the top corners with the corresponding reference horizontal line and / or reference vertical line, and calculating the offset amount of each top corner that needs to be corrected; and pre-correcting the page image to be corrected according to the calculated offset amount.

[0106] Optionally, the top corners in the page image to be corrected can be automatically identified by a trained artificial intelligence model, and the position information of the identified top corners can be stored in a database.

[0107] Optionally, the step of pre-correcting the page includes: removing irrelevant information in the page image to be corrected, and stretching the page area in the page image to be corrected to a standardized shape corresponding to the page.

[0108] Optionally, the step of pre-correcting the page according to the calculated relative offset amount further includes:

[0109] The relative offset amount of the pre-correction auxiliary point located between any two top corners is calculated by interpolation or curve fitting.

[0110] For the case where the edges of the paper are not photographed and the four top corner information of the page where the paper is located cannot be determined, the four top corners of the page image to be corrected are regarded as the four top corners of the paper page, and the page pre-correction is no longer performed.

[0111] The application also discloses a page distortion correction system, which specifically comprises:

[0112] A check point selection unit is configured to determine a target text line to be corrected, and select a plurality of check points in the target text line, wherein the plurality of check points satisfy that the connecting line of the plurality of check points coincides with or is parallel to the correction reference line when the page is not distorted.

[0113] An offset amount determination unit is configured to compare the positions of the plurality of check points with the correction reference line, and calculate the offset amount of the check points that needs to be corrected.

[0114] A correction unit is configured to correct each text of the target text line according to the calculated offset amount.

[0115] The target text line refers to the text arranged in a row or a column in the current page image, including the target text column arranged in a column, because the row and the column are only different in arrangement direction, the essence of both is the text array arranged in a line.

[0116] The check point selection unit determines the target text line to be corrected, and also performs the operation of identifying the text line in the page image to be corrected.

[0117] The manner of recognizing the text line in the page image to be corrected comprises any one or a combination of the following manners:

[0118] detecting a text target in the page image to be corrected and obtaining position information of the text target; determining a target text line located on the same line based on the position information of the text target; or,

[0119] recognizing the text line in the page image to be corrected according to image grayscale of the page image to be corrected, and determining the text line if there is a horizontally extended grayscale area in the page image to be corrected and the width meets a preset value; or,

[0120] detecting the text line by using an artificial intelligence model for detecting a text line in an image.

[0121] The manner of detecting a text target in the page image to be corrected comprises any one or a combination of the following manners:

[0122] detecting the text target by recognizing a distorted part in the page image to be corrected and judging whether the recognized content contains representative features of the text, and obtaining position information of the corresponding text target when it is judged that the recognized content contains the representative features of the text, wherein the representative features of the text include strokes, radicals, characters and letters;

[0123] detecting the text target by performing OCR recognition on the page image to be corrected and obtaining position information of the recognized text target;

[0124] detecting the text target by using an artificial intelligence model for detecting a text target in an image, and obtaining position information of the detected text target.

[0125] The checkpoint selection unit selects a point at a set proportion position based on the font size of the target text line when selecting a plurality of checkpoints for the target text line.

[0126] Preferably, a point at a set proportion position in the font width and / or height direction of the text is selected as a checkpoint; for example, a center position of the width and height of the text, a one-third high and one-half wide position, a two-thirds high and two-thirds wide position, and the like. The position may or may not have font strokes, which does not affect the realization of the final purpose as long as the coordinate value can be obtained. The specific coordinate calculation process can use known mathematical methods, because Chinese characters are square characters, and even if there is stretching, distortion and deformation, they can still be regarded as rough parallelograms or trapezoids, and the point at the specific proportion position can be approximately calculated according to the proportion. Since Chinese characters account for a small proportion of the entire page, even if there is an error, the final effect is less affected.

[0127] Preferably, the target text is printed text, because printed text output is stable, unlike handwritten text which may have defects such as font size changing, font row and column tilting in a certain direction, etc. when the page is not distorted.

[0128] Optionally, the correction unit further performs the following operation: correcting the text in the adjacent row of the target text row according to the calculated offset;

[0129] Optionally, the correction unit further performs the following operation: correcting the area consistent with the distortion of the target text row according to the calculated offset;

[0130] Optionally, each target text row to be corrected corresponds to a correction reference line, and a plurality of correction reference lines on the same page to be corrected are arranged in parallel and at equal intervals.

[0131] Optionally, the correction unit further performs the following operation: performing the above operation on all target text rows in the page to be corrected, so as to take the selected check points on the target text row as anchor points to correct the entire page to be corrected.

[0132] In the above embodiment, the correction reference line is a horizontal line corresponding to the target text row, and for the target text column, it is a vertical line.

[0133] In the above embodiment, when the offset determining unit compares the positions of the check points with the correction reference line to calculate the offset that the check points need to be corrected, the leftmost or rightmost point is taken as the reference point, and the relative offset in the longitudinal direction of all check points is calculated relative to the reference point.

[0134] Further, the operation of the offset determining unit to calculate the offset that the check points need to be corrected can further include: comparing the positions of the check points with the theoretical position of each character calculated according to the width of the target text row and the number of characters in the row, to obtain the relative offset of all check points in the transverse direction.

[0135] In the present application, the vertical offset is the distance between the vertical coordinate value of the font to which the other point belongs and the correction reference line passing the reference point; and the horizontal offset can be estimated according to the width of the target text line and the number of characters in the line, for example. The following is an example. In the image of the page to be corrected, it is assumed that the width of the page is 1000 pixels, the normal width of the target text line should be 840 pixels, and the distance from the leftmost end of the leftmost character to the rightmost end of the rightmost character in the target text line is reduced to 640 pixels due to the distortion, and the target text line contains 20 characters. Since the corrected characters should be arranged in equal width and equal character spacing (the advantage of square characters), 20 characters, 19 spaces, approximately 20 characters and 20 spaces, the distance of the first character from the reference point of the leftmost character can be approximately calculated according to the corresponding proportion of the width of the standard text line. The horizontal offset is the relative offset of the actual horizontal coordinate value of the check point relative to the distance.

[0136] In the correction, the font to which the reference point belongs is not moved, and in the vertical direction, the font to which the other point belongs is moved to a line passing the reference point and parallel to the correction reference line; and in the horizontal direction, the font to which the other point belongs is moved relative to the reference point according to the horizontal offset.

[0137] Preferably, the offset determination unit further evaluates the distortion degree of the page to be corrected before performing the operation of calculating the offset that needs to be corrected for the check point.

[0138] Further preferably, the step of evaluating the distortion degree of the page to be corrected is implemented by comparing the offset with the width and height of the font in the target text line, for example; other lengths can also be used as references, such as the width and height of the entire target text line. The reference object is different, and the evaluation score is different.

[0139] Further preferably, when the evaluation score of the distortion degree of the page to be corrected is lower than a first threshold value, such as 5%, 8% or 10%, the correction program is ended. The specific value of the first threshold value can be determined according to the task requirements or experience value.

[0140] Optionally, when the offset determination unit calculates the offset that needs to be corrected for the check point, it also calculates the offset of the check auxiliary point between the two selected check points by interpolation or curve fitting; and subsequently corrects the characters between the two selected check points according to the offset of the check auxiliary point.

[0141] The page distortion correction system further comprises a page pre-correction unit for pre-correcting the page before the check point selection unit determines the target text line to be corrected.

[0142] Optionally, the page pre-correction unit pre-corrects the page by: determining the top corners of the paper page from the page image to be corrected; comparing the coordinates of the top corners with the corresponding reference horizontal line and / or reference vertical line, and calculating the offset of each top corner to be corrected; and pre-correcting the page image to be corrected according to the calculated offset.

[0143] Optionally, the top corners in the page image to be corrected can be automatically identified by a trained artificial intelligence model, and the position information of the identified top corners can be stored in a database.

[0144] Optionally, when the page pre-correction unit pre-corrects the page, the following operations are performed: irrelevant information in the page image to be corrected is removed, and the page region in the page image to be corrected is stretched to a standardized shape corresponding to the page.

[0145] Optionally, when the page pre-correction unit pre-corrects the page image to be corrected according to the calculated relative offset, the following operations are performed:

[0146] The relative offset of the pre-correction auxiliary point located between any two top corners is calculated by interpolation or curve fitting.

[0147] Optionally, for the case where the edges of the paper are not photographed and the four top corner information of the page where the paper is located cannot be determined, the four top corners of the page image to be corrected are regarded as the four top corners of the paper page, and the page pre-correction is no longer performed.

[0148] As shown in Figure 3 The present application also discloses an electronic device comprising a processor and a memory, wherein the memory is used to store a computer executable program, and when the computer executable program is executed by the processor, the processor executes the method for page distortion correction as described above.

[0149] The electronic device can be in the form of a general computing device. The processor can be one or multiple and work cooperatively. The present application does not exclude distributed processing, i.e., the processor can be dispersed in different physical devices. The electronic device of the present application is not limited to a single physical entity, but can also be the sum of multiple physical devices.

[0150] The memory stores a computer executable program, which is usually machine readable code, and the computer executable program can be executed by the processor to enable the electronic device to execute the method of the present application or at least part of the steps in the method.

[0151] The memory comprises volatile memory, such as random access memory (RAM) and / or cache memory, and can also comprise non-volatile memory, such as read-only memory (ROM).

[0152] Optionally, the electronic device further comprises an I / O interface for data exchange between the electronic device and external devices. The I / O interface can be one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0153] The electronic device of the present application can also include elements or components not shown in the above examples. For example, some electronic devices further comprise a display unit such as a display screen, and some electronic devices further comprise human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute the computer-readable program in the memory to implement the method or at least part of the steps of the method of the present application, it can be considered as an electronic device covered by the present application.

[0154] As shown in Figure 4 The present application also discloses a computer-readable medium having a computer-executable program stored thereon, wherein the computer-executable program, when executed, implements the page distortion correction method as described above.

[0155] The computer-readable storage medium can include a data signal carried in a baseband or as a part of a carrier wave, in which readable program codes are borne. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable storage medium can also be any medium that can be read by a machine or a computer, which can send, transmit, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program codes contained in the computer-readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0156] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Python, Java, C++, etc., and a conventional procedural programming language such as C, assembly language, or a similar programming language. The program code can be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, through the Internet using an Internet service provider).

[0157] The application also discloses a computer program, which, when executed by a computer, implements the page distortion correction method.

[0158] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and in conjunction with specific embodiments. It should be noted that the following embodiments are only used to illustrate the present application, but not to limit the present application.

[0159] Embodiment 1

[0160] The page distortion correction method provided by the embodiment is generally divided into the following steps:

[0161] 1. Locating the page by detecting the four corners of the page;

[0162] 2. Preliminary correction (rough correction, which can shield some interference for the following correction) of the page by the four corners;

[0163] 3. Locating some check points by a certain scheme, for example, the set proportion of the font;

[0164] 4. Determining the correction parameters of some discrete points in the picture according to the check points;

[0165] 5. The correction amplitude of all pixel points on the whole picture is obtained by interpolation according to the correction amplitude of the known discrete points, and then the final correction effect of the whole page can be obtained.

[0166] Embodiment 2

[0167] The page distortion correction system of the embodiment is pre-installed on a mobile phone, and specifically comprises:

[0168] A check point selection unit is configured to determine a target text line to be corrected and select a plurality of check points in the target text line, wherein the plurality of check points satisfy that a line connecting the plurality of check points is coincident with or parallel to a correction reference line when the page is not distorted.

[0169] An offset determination unit is configured to compare the positions of the plurality of check points with the correction reference line and calculate offset amounts of the check points to be corrected.

[0170] A correction unit is configured to correct each text of the target text line according to the calculated offset amounts.

[0171] Figure 5 is a schematic diagram of calculating the position coordinates of the check points of the text in line in Embodiment 2 of the present application, Figure 6 is an enlarged schematic diagram of calculating the position coordinates of the check points of some specific texts. As an example, as shown inFigure 5 , 6 as shown in FIG. 6B, Figure 5 The middle row in FIG. 6B shows the coordinates of the middle points of the width and height of the target text line as the check points (thick black dots), and the leftmost check point is taken as the reference point. The vertical offset of each check point relative to the correction reference line (dashed line) is determined by comparing the vertical coordinates of the check points with the correction reference line. The horizontal offset of each check point relative to the reference point is determined by comparing the horizontal coordinates of the check points with the distance calculated according to the width of the target text line and the number of characters in the line. In FIG. 6B, the first check point is taken as the reference point, and the subsequent check points are moved in the direction of the exemplary arrow (the horizontal movement direction is not shown in the figure). Thus, the target text line is rearranged into a straight line, and the correction of all the characters in the target text line is completed.

[0172] The present application can also perform the same operation on the entire page image (possibly including pictures) according to the calculated offset of each check point, taking the offset as an anchor point, so as to realize the correction of the entire page image.

[0173] Figure 5 The lower row in FIG. 6B shows the curve fitting of the coordinates of the middle points of the width and height. Although the position coordinates of the check points are not greatly offset, the calculated offset of the image coordinates between two check points is more accurate than the simple interpolation, because the trend change is considered in the curve fitting process, instead of the simple addition and average.

[0174] Figure 9 FIG. 7 is a photo of the actual processing effect of the embodiment 2 of the present application. As can be seen from the figure, the characters in the left page are slightly raised upward due to the slight upward bending of the paper, and are basically arranged in a straight line after the processing of the embodiment. Since the scheme of the embodiment 2 of the present application does not need to compare with the font coordinates in the standard font library, the judgment process of comparing with several points of the font coordinates can be simplified, and the comparison efficiency and operation speed are improved.

[0175] Embodiment 3

[0176] The embodiment is basically the same as the embodiment 2, and the difference is that it further includes a page pre-correction unit for performing a page pre-correction operation before the step of performing page correction. The page pre-correction operation specifically includes:

[0177] A page corner recognition sub-unit is configured to determine the top corner of the paper page from the page image to be corrected. Since the paper may be curled and bent, the corresponding edges are formed into inclined lines, curved lines or irregular lines, but the top corner defined by the two edges is relatively easy to distinguish and locate.

[0178] a page distortion offset calculation subunit for comparing the coordinates of the four corners with a reference horizontal line and / or a reference vertical line, and calculating the relative offset that each corner needs to be corrected;

[0179] a page distortion correction subunit for pre-correcting the page image to be corrected according to the calculated relative offset.

[0180] Figure 7 is a schematic photo of the pre-correction of the four corners of Embodiment 3 of the present application, in which the bottom left corner slightly exceeds the page, and since there is no text in this part, it can be regarded as the actual corner of the paper, and the page pre-correction step of the present application is used for pre-correction. The square in the photo indicates that the correction only considers the positions of the four corners, and does not consider the situation in which the curled edge exceeds the range of the corner, but the correction effect after interpolation calculation is still satisfactory.

[0181] Figure 8 is a situation in which the two bottom corners are outside the shooting frame, and part of the text is also lost, at this time, the page pre-correction step of the present application is no longer suitable for page pre-correction, and feedback information is suggested to the customer to re-shoot, or the next step of page correction program is directly executed.

[0182] Through the description of the above embodiments, those skilled in the art can easily understand that the page distortion correction scheme of the present application can quickly correct the distortion of various pages / characters caused by the deformation of the bearing plane, and has good restoration; in addition, the correction method of the present application has strong adaptability, and is suitable for the situation of character distortion caused by the bending and wrinkling of the bearing paper, and the correction result is more stable, and the case of bad result is not easy to appear.

[0183] The above-described specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application, and it should be understood that the present application is not inherently related to any specific computer, virtual device or electronic device, and various general-purpose devices can also implement the present application. The above description is only for specific embodiments of the present application and is not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method of page skew correction, characterized by, The method comprises the following steps: Pre-correcting the page to remove irrelevant information in the page image to be corrected and to stretch the page region in the page image to be corrected to a normalized shape corresponding to the page, and the page pre-correction specifically comprises: determining the top corners of the paper page from the page image to be corrected; comparing the coordinates of the top corners with the corresponding reference horizontal line and / or reference vertical line to calculate the offset amount of correction required by each top corner; and pre-correcting the page image to be corrected according to the calculated offset amount; Determining the target text line to be corrected currently and selecting a plurality of check points in the target text line, comprising: setting a proportional equidistant sampling of a plurality of check points in the font width and / or height direction of the text based on the length and font size of the target text line, and the plurality of check points satisfy: the plurality of check points are located on the same straight line when the page is not distorted and the connecting line of the plurality of check points coincides or is parallel to the correction reference line, and the correction reference line is the horizontal line corresponding to the target text line; wherein the correction reference line is a parallel line to the edge line of the image and is located at the middle position of the target text line to be corrected currently, and each target text line to be corrected on the page corresponds to a correction reference line, and the plurality of correction reference lines on the same page are parallel to each other and arranged equidistantly; Comparing the positions of the plurality of check points with the correction reference line and calculating the offset amount of correction required by the check points relative to the reference point, comprising: taking the leftmost or rightmost check point as the reference point, and calculating the relative offset amount in the longitudinal direction of all check points relative to the reference point, and comparing the positions of the plurality of check points with the theoretical position of each character calculated according to the width and number of lines of text of the target text line to obtain the relative offset amount in the horizontal direction of all check points; wherein the longitudinal offset amount is the distance between the longitudinal coordinate value of the font of the check point outside the reference point and the correction reference line passing through the reference point, and the horizontal offset amount is the deviation of the actual horizontal coordinate value of the check point from the distance between the corrected text and the reference point; Evaluating the distortion degree of the page to be corrected by comparing the offset amount with the width and height of the font in the target text line; Correcting each character of the target text line according to the calculated offset amount, comprising: for longitudinal direction correction, the font of the reference point does not move and the fonts of other points are moved to a line passing through the reference point and parallel to the correction reference line; for horizontal direction correction, the font of the reference point does not move and the fonts of other points are moved according to the relative offset amount; the text at the check point is moved as a whole, and each character of the target text line to be corrected currently is arranged in a straight line. The operations of selecting a plurality of check points, comparing and calculating offset amount, comparing and evaluating, and correcting are performed on all target text lines in the page to be corrected, and the selected check points on the target text line are used as anchor points to correct the entire page to be corrected currently.

2. The method of claim 1, wherein, Further comprising: Identifying the text line in the page image to be corrected, and the identification is performed by one or more of the following methods in combination: detecting a text target in the page image to be corrected and obtaining position information of the text target; determining a text line located on the same line based on the position information of the text target; or, identifying a text line in the page image to be corrected according to image grayscale of the page image to be corrected, and determining that the page image to be corrected is a text line if there is a horizontally or vertically extended grayscale area in the page image to be corrected and the width of the grayscale area meets a preset value; or detecting the text line by using an artificial intelligence model for detecting a text line in an image.

3. The method of claim 2, wherein, The detecting a text target in the page image to be corrected and obtaining position information of the text target includes any one or a combination of the following manners: detecting the text target by performing OCR recognition on the page image to be corrected, and obtaining position information of the text target recognized; detecting the text target by using an artificial intelligence model for detecting a text target in an image, and obtaining position information of the text target detected.

4. The method of claim 1, wherein, The sampling of the check points includes: selecting points at set proportion positions in the font width and / or height direction of the text as the check points; and / or, the target text is a printed font.

5. The method of claim 1, wherein, Further includes: correcting the text of the adjacent line of the target text line according to the offset calculated; and / or, correcting the area consistent with the distortion of the target text line according to the offset calculated. The evaluating the distortion degree of the page to be corrected further includes:

6. The method of claim 1, wherein, when the evaluation score of the distortion degree of the page to be corrected is lower than a first threshold value, ending the correction program; and / or, The calculating the offset of the check points that need to be corrected further includes: calculating the offset of a check auxiliary point on the text between two selected check points by interpolation or curve fitting; and correcting each text of the target text line according to the offset of the check auxiliary point, specifically including: correcting the text between the two selected check points according to the offset of the check auxiliary point. The determining the top corners of the paper page from the page image to be corrected includes: automatically identifying the top corners by using a trained artificial intelligence model and storing the position information of the top corners identified in a database.

7. The method of claim 1, wherein, The pre-correcting the page image to be corrected according to the relative offset calculated further includes:

8. The method of claim 7, wherein, calculating the relative offset of a pre-correction auxiliary point located between any two top corners by interpolation or curve fitting. It includes:

9. A page skew correction system characterized by, a page pre-correction unit, which pre-corrects the page to remove irrelevant information in the page image to be corrected and stretches the page area in the page image to be corrected to a standardized shape corresponding to the page, and the page pre-correction specifically includes: determining the top corners of the paper page from the page image to be corrected; comparing the coordinates of the top corners with corresponding reference horizontal lines and / or reference vertical lines, and calculating the offset of each top corner that needs to be corrected; and pre-correcting the page image to be corrected according to the offset calculated; ​ The check point selection unit determines a target text line to be corrected and selects a plurality of check points in the target text line, including: setting a proportional equidistant sampling of a plurality of check points in the font width and / or height direction based on the length and font size of the target text line, and the plurality of check points satisfy: the plurality of check points are located on the same straight line when the page is not distorted, and the connecting line of the plurality of check points coincides or is parallel to the correction reference line, and the correction reference line is a horizontal line corresponding to the target text line; wherein the correction reference line is a parallel line to the edge line of the image and is located at the middle position of the current target text line to be corrected, and each line of the target text line to be corrected on the page to be corrected corresponds to a correction reference line, and a plurality of correction reference lines on the same page to be corrected are parallel to each other and arranged at equal intervals. The offset determination unit compares the positions of the plurality of check points with the correction reference line and calculates the offset of the check points that need to be corrected relative to the reference point, including: taking the leftmost or rightmost check point as the reference point, and calculating the relative offset of all check points in the vertical direction relative to the reference point, and comparing the positions of the plurality of check points with the theoretical position of each character calculated according to the width and number of characters of the target text line to obtain the relative offset of all check points in the horizontal direction; wherein the vertical offset is the distance between the vertical coordinate value of the check point outside the reference point and the correction reference line passing through the reference point, and the horizontal offset is the distance between the actual horizontal coordinate value of the check point and the distance between the corrected character and the reference point. The correction unit evaluates the distortion degree of the page to be corrected by comparing the offset with the width and height of the font in the target text line; corrects each character of the target text line according to the calculated offset, including: when correcting in the vertical direction, the font of the reference point does not move, and the fonts of other points are moved to a line passing through the reference point and parallel to the correction reference line; when correcting in the horizontal direction, the font of the reference point does not move, and the fonts of other points are moved according to the relative offset; the characters at the check points are moved as a whole, and each character of the current target text line to be corrected is arranged in a straight line; and the operations of selecting a plurality of check points, comparing and calculating offsets, comparing and evaluating, and correcting are performed on all target text lines in the page to be corrected, and the selected check points on the target text line are used as anchor points to correct the entire current page to be corrected.

10. An electronic device comprising a processor and a memory, the memory being configured to store a computer executable program, characterized in that: When the computer executable program is executed by the processor, the processor executes the method of any one of claims 1-8.

11. A computer readable medium storing a computer executable program, characterized in that, The computer executable program is executed to implement the method of any one of claims 1-8.

Images