Text image processing method and apparatus

Abstract

Embodiments of the present disclosure provide a text image processing method and device. The text image processing method comprises: obtaining a first feature vector according to a plurality of first coordinates of each of N text units; the N text units are in a first text image; the first feature vector has N rows and X columns, and one row of the first feature vector corresponds to one text unit; N is a positive integer, and X is an integer greater than or equal to 2; correcting the first feature vector according to features of different columns of the first feature vector to obtain a second feature vector; the second feature vector has N rows and X columns, wherein one row of the second feature vector corresponds to one text unit; fusing the first feature vector and the second feature vector to obtain a third feature vector, the third feature vector has N rows and Y columns; wherein one row of the third feature vector is used for position correction of one text unit; and correcting the first text image according to the third feature vector to obtain a second text image.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of image processing technology, and in particular to a text image processing method and apparatus. Background Technology

[0002] When capturing images of printed text using image acquisition devices, the shape of the printed text and the image acquisition angle can cause text lines in the captured image to appear curved or tilted, affecting the user's ability to view the text content. For example, when capturing images of a thick book, the resulting text image may appear curved.

[0003] In related technologies, text image processing methods such as perspective transformation, polynomial mapping, and grid transformation can be used to correct text images with distorted text lines, but the correction effect is not ideal. Summary of the Invention

[0004] This disclosure provides a text image processing method and apparatus.

[0005] The first aspect of this disclosure provides a text image processing method, the method comprising:

[0006] A first feature vector is obtained based on multiple first coordinates of each of the N text units; the N text units are in the first text image; the first feature vector has N rows and X columns, and one row of the first feature vector corresponds to one text unit; N is a positive integer, and X is an integer greater than or equal to 2;

[0007] The first feature vector is corrected based on the features of different columns of the first feature vector to obtain a second feature vector; the second feature vector has N rows and X columns, wherein one row of the second feature vector corresponds to one text unit;

[0008] The first feature vector and the second feature vector are fused to obtain a third feature vector, which has N rows and Y columns; wherein, each row of the third feature vector is used for the position correction of one text unit;

[0009] The first text image is corrected based on the third feature vector to obtain the second text image.

[0010] Optionally, one of the text units includes M of the first coordinates;

[0011] The step of obtaining the first feature vector based on multiple first coordinates of each of the N text units includes:

[0012] Extract the outer contour corresponding to the first coordinate line of the nth text unit to obtain the nth row of the first feature vector; where n is a positive integer less than or equal to N.

[0013] Optionally, a plurality of the first coordinates of a text unit includes: end coordinates and intermediate coordinates located between the end coordinates, the end coordinates being located at both ends of the text unit;

[0014] The method further includes:

[0015] Based on the text distribution of each text unit in the first text image, obtain the end coordinates of each text unit;

[0016] Based on the text distribution of each text unit, one or more intermediate coordinates are collected between any two end coordinates.

[0017] Optionally, the step of acquiring one or more intermediate coordinates between any two end coordinates based on the text distribution of each text unit includes:

[0018] The first text image is segmented by N first text boxes, wherein one first text box is used to define one text unit;

[0019] Based on the position of the vertex of the nth first text box, determine the end coordinates of the nth text unit;

[0020] One or more of the intermediate coordinates are collected on the border of the nth first text box.

[0021] Optionally, the method further includes:

[0022] The number of intermediate coordinates collected on the border of the nth first text box is determined based on the curvature and / or slope of the outer contour of the nth first text box.

[0023] Optionally, the step of fusing the first feature vector and the second feature vector to obtain the third feature vector includes:

[0024] The first feature vector and the second feature vector are added together to obtain the third feature vector.

[0025] Optionally, obtaining the first feature vector based on multiple first coordinates of each of the N text units includes:

[0026] Based on the plurality of first coordinates included in each of the N text units, a first matrix is ​​obtained, wherein the first matrix has N rows and M columns, one row of the first matrix corresponds to one text unit, and M is the number of first coordinates included in one text unit;

[0027] Interpolation is performed on each row of the first matrix to obtain the first feature vector.

[0028] Optionally, the step of interpolating each row of the first matrix to obtain the first feature vector includes:

[0029] The first matrix is ​​normalized to obtain the second matrix;

[0030] Interpolation is performed on each row of the second matrix to obtain the first feature vector.

[0031] Optionally, correcting the first text image based on the third feature vector to obtain the second text image includes:

[0032] Perform at least one convolution and at least one pooling operation on the third feature vector to obtain the second coordinates after correcting the first coordinates;

[0033] The second text image is generated based on the text content of the first text image and the second coordinates.

[0034] A second aspect of this disclosure provides a text image processing apparatus, the apparatus comprising:

[0035] The module is configured to obtain a first feature vector based on multiple first coordinates of each of the N text units; the N text units are in a first text image; the first feature vector has N rows and X columns, and one row of the first feature vector corresponds to one of the text units; N is a positive integer, and X is an integer greater than or equal to 2;

[0036] The first correction module is used to correct the first feature vector according to the features of different columns of the first feature vector to obtain a second feature vector, wherein the second feature vector has N rows and X columns, and one row of the second feature vector corresponds to one text unit;

[0037] A fusion module is used to fuse the first feature vector and the second feature vector to obtain a third feature vector, the third feature vector having N rows and Y columns; wherein, one row of the third feature vector is used for the position correction of one text unit;

[0038] The second correction module is used to correct the first text image based on the third feature vector to obtain the second text image.

[0039] Optionally, one of the text units includes M of the first coordinates;

[0040] The obtained module is used for:

[0041] Extract the outer contour corresponding to the first coordinate line of the nth text unit to obtain the nth row of the first feature vector, where n is a positive integer less than or equal to N.

[0042] Optionally, a plurality of first coordinates of a text unit includes: end coordinates and intermediate coordinates located between the end coordinates, the end coordinates being located at both ends of the text unit;

[0043] The device further includes:

[0044] The acquisition module is used to acquire the end coordinates of each text unit based on the text distribution of each text unit in the first text image;

[0045] The setting module is used to collect one or more intermediate coordinates between any two end coordinates based on the text distribution of each text unit.

[0046] Optionally, the acquisition module is used to:

[0047] The first text image is segmented by N first text boxes, wherein one first text box is used to define one text unit;

[0048] Based on the position of the vertex of the nth first text box, determine the end coordinates of the nth text unit;

[0049] One or more of the intermediate coordinates are collected on the border of the nth first text box.

[0050] Optionally, the device further includes:

[0051] The determination module is used to determine the number of intermediate coordinates collected on the border of the nth first text box based on the curvature and / or slope of the outer contour of the nth first text box.

[0052] Optionally, the fusion module is used for:

[0053] The first feature vector and the second feature vector are added together to obtain the third feature vector.

[0054] Optionally, the obtaining module is further configured to:

[0055] Based on the plurality of first coordinates included in each of the N text units, a first matrix is ​​obtained, wherein the first matrix has N rows and M columns, one row of the first matrix corresponds to one text unit, and M is the number of first coordinates included in one text unit;

[0056] Interpolation is performed on each row of the first matrix to obtain the first feature vector.

[0057] Optionally, the obtaining module is further configured to:

[0058] The first matrix is ​​normalized to obtain the second matrix;

[0059] Interpolation is performed on each row of the second matrix to obtain the first feature vector.

[0060] Optionally, the second correction module is used for:

[0061] Perform at least one convolution and at least one pooling operation on the third feature vector to obtain the second coordinates after correcting the first coordinates;

[0062] The second text image is generated based on the text content of the first text image and the second coordinates.

[0063] A third aspect of this disclosure provides an electronic device, comprising:

[0064] processor;

[0065] Memory used to store processor-executable instructions;

[0066] The processor is configured to implement the method provided by any one of the technical solutions in the first aspect when running the executable instructions.

[0067] A fourth aspect of this disclosure provides a computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the method provided by any of the solutions in the first aspect described above.

[0068] The technical solution provided in this disclosure obtains a first feature vector based on the first coordinates of N text units in a first text image. The first feature vector is then corrected based on the features of different columns to obtain a second feature vector. Finally, the first and second feature vectors are fused to obtain a third feature vector. Since the multiple first coordinates of each text unit reflect its shape, and the second feature vector indicates the positional relationship between text units, correcting the text units in the first text image using the third feature vector obtained by fusing the first and second feature vectors can correct deformed text lines into horizontal text lines and deformed text columns into vertical text columns without causing confusion in the positional relationship between text lines or columns, thus improving the quality of the corrected second text image.

[0069] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0071] Figure 1 This is a schematic flowchart of a text image processing method provided in an embodiment of the present disclosure.

[0072] Figures 2 to 6 A schematic diagram of a first text image and a second text box provided for embodiments of this disclosure.

[0073] Figure 7 This is a schematic diagram of the structure of the target model provided in an embodiment of this disclosure.

[0074] Figure 8 This is a schematic diagram illustrating the process of training a target model according to an embodiment of this disclosure.

[0075] Figure 9 This is a schematic diagram illustrating the process of testing a target model according to an embodiment of this disclosure.

[0076] Figure 10 This is a schematic flowchart of another text image processing method provided in an embodiment of the present disclosure.

[0077] Figure 11 This is a schematic flowchart of a text image processing apparatus provided in an embodiment of the present disclosure.

[0078] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this disclosure as detailed in the appended claims.

[0080] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0081] like Figure 1 As shown, this disclosure provides a text image processing method, which includes:

[0082] S101: Obtain the first feature vector based on the multiple first coordinates of each of the N text units; the N text units are in the first text image; the first feature vector has N rows and X columns, and one row in the first feature vector corresponds to one text unit; N is a positive integer, and X is an integer greater than or equal to 2;

[0083] S102: Correct the first feature vector according to the features of different columns of the first feature vector to obtain the second feature vector; the second feature vector has N rows and X columns, wherein one row in the second feature vector corresponds to one text unit;

[0084] S103: Merge the first and second feature vectors to obtain the third feature vector, which has N rows and Y columns; where each row of the third feature vector is used for the position correction of a text unit;

[0085] S104: Correct the first text image based on the third feature vector to obtain the second text image.

[0086] The text image processing method of this disclosure can be executed by an electronic device having an image acquisition component, or by an electronic device capable of establishing a connection with an image acquisition device.

[0087] For example, the electronic device used to perform the image processing method of the embodiments of this disclosure may be a smartphone, tablet computer, laptop computer, desktop computer or other electronic device with computing capabilities.

[0088] In this embodiment of the disclosure, the first text image to be processed can be a text image in which the shape of the text units in the image is deformed.

[0089] For example, the first text image in this disclosure embodiment may be a text image in which at least one text line or at least one text column is deformed such as bending and / or tilting.

[0090] For example, in this embodiment of the disclosure, the first text image to be processed can be as follows: Figure 2 The left half of the middle Figure 3 The left half or Figure 5 The left half of the image shows a text image where the text line appears distorted and warped. For ease of explanation, the text image with distorted text lines or text lists will be referred to as a warped text image in the following text.

[0091] For example, the first text image to be processed in this embodiment of the present disclosure may also be as follows: Figure 4 The left half or Figure 6 The left half of the image shows a text image where the text line appears slanted. For ease of explanation, the text image where the text line or text column appears slanted will be referred to as a slanted text image in the following text.

[0092] In some embodiments, the first text image may be an image of collected paper text or a scanned document.

[0093] In some embodiments, if the text content in the first text image is arranged in rows, then one text row can be taken as a text unit to obtain N text units of the first text image. If the text content in the first text image is arranged in columns, then one text column can be taken as a text unit to obtain N text units of the first text image.

[0094] In this embodiment of the disclosure, before obtaining the first feature vector based on the multiple first coordinates of each of the N text units in the first text image, it is necessary to first obtain the first coordinates of each text unit in the first text image.

[0095] In one embodiment, it can be constructed Figure 2 The first coordinate system shown is defined as follows: the origin of the first coordinate system is the upper left corner of the first text image; the positive direction of the X-axis is horizontally to the right of the first text image; and the positive direction of the Y-axis is vertically downwards of the first text image. Subsequently, coordinate values ​​of multiple points are collected from each text unit, and these coordinate values ​​are used as the first coordinates of the corresponding text unit.

[0096] For example, coordinates can be collected from multiple points in each text unit of the first text image at preset intervals to obtain the first coordinates of each text unit.

[0097] The preset interval can be dynamically set as needed, and this embodiment does not limit it.

[0098] For example, the preset interval can be set statically, and the preset interval can be set to any value between 0.2cm and 2cm.

[0099] In another embodiment, the coordinates of the text units of the first text image can also be the pixel coordinates of multiple points in each text unit in the first text image.

[0100] That is, obtain the pixel coordinates of multiple points in each text unit in the first text image, and use the pixel coordinates obtained from each text unit as the first coordinate of the corresponding text unit.

[0101] In some embodiments, each text unit may include more than four first coordinates, and exemplaryly, the number of first coordinates may be an even number of more than four, so that the first coordinates may reflect the shape of the outer contour of the text unit in the first text image.

[0102] In another embodiment, the first text image can be input into the text detection network to obtain the first coordinates output by the text detection network.

[0103] The text detection network described above can be any network capable of performing coordinate detection on each text unit of a text image, and this embodiment of the disclosure does not limit it.

[0104] For example, the text detection network mentioned above can be DBnet (Deep Bidirectional network) or CTPN network (Connectionist Text Proposal Network).

[0105] In some embodiments, when a text line in the first text image is a text unit, the coordinate values ​​of multiple points can be collected from each text line, and the coordinate values ​​of the collected multiple points are used as multiple first coordinates included in the corresponding text unit.

[0106] In other embodiments, when a text column in the first text image is a text unit, the coordinate values ​​of multiple points can be collected from each text column, and the coordinate values ​​of the collected multiple points can be used as multiple first coordinates included in the corresponding text unit.

[0107] In some embodiments, correcting the first feature vector based on the features of different columns of the first feature vector to obtain the second feature vector includes:

[0108] The first feature vector is input into a Long Short Term Memory (LSTM) network, and the second feature vector is output by the LSTM network after correcting the first feature vector based on the features of different columns in the first feature vector.

[0109] For example, the features of different columns in the first feature vector include, but are not limited to: the relative position features between multiple columns included in the same text unit, and the relative position features between the same columns in different text units.

[0110] In some embodiments, a text line of a first text image refers to an image of a line of characters in a printed text within the first text image, and a text column of a first text image refers to an image of a column of characters in a printed text within the first text image.

[0111] For example, if the first text image is an image of a page of a book, then a text action is an image of a line of text in the book within the first text image, and a text column is an image of a column of text in the book within the first text image.

[0112] In one embodiment, if a text line in the first text image is a text unit, then the text content of each text line in the second text image obtained after correcting the first text image is arranged horizontally, and adjacent text lines are equally spaced vertically. If a text column in the first image is a text unit, then the text content of each text column in the second text image obtained after correcting the first text image is arranged vertically, and adjacent text columns are equally spaced horizontally.

[0113] For example, the text lines in the second text image can be arranged as follows: Figures 2 to 6 The arrangement shown in the right half of the image.

[0114] In this embodiment, a first feature vector is obtained based on multiple first coordinates of each text unit in the first text image. The first feature vector is then corrected based on the features of different columns to obtain a second feature vector. Finally, the first and second feature vectors are fused to obtain a third feature vector. Since the column coordinates of the text image change when the image is deformed, the features of different columns of the first feature vector can, to some extent, reflect the deformation trend of each text unit in the first text image. Therefore, correcting the first feature vector based on the features of different columns yields a second feature vector that reflects the shape characteristics of the corrected text unit. Thus, correcting the text units in the first text image using the third feature vector obtained by fusing the first and second feature vectors can correct deformed text units into horizontal or vertical text units, improving the quality of the corrected second text image.

[0115] In one embodiment, S101 includes:

[0116] Extract the outer contour corresponding to the first coordinate line of the nth text unit to obtain the nth row of the first feature vector; n is a positive integer less than or equal to N.

[0117] In this embodiment of the disclosure, after obtaining the first text image to be corrected, the coordinates of M points can be collected from each text unit of the first text image, where M is an integer greater than or equal to 4.

[0118] For example, a first text image can be input into a text detection network, the edge region of each text unit can be detected by the text detection network, and the position region of the text unit in the first text image can be determined based on the detected edge region of each text unit. Then, for the nth text unit in the first text image, the coordinates of M points can be collected from the position region where the nth text unit is located, and the coordinates of the collected M points can be used as the first coordinates included in the nth text unit.

[0119] If the text unit is a text line, when collecting the first coordinates of the nth text line, the coordinates of M points can be collected from the middle area of ​​the nth text line along the extension direction of the text content of the nth text line, or the coordinates of M points can be collected from the top or bottom area of ​​the nth text line along the extension direction of the text content of the nth text line.

[0120] If the text unit is a text column, when collecting the first coordinates of the nth text column, the coordinates of M points can be collected from the middle area of ​​the nth text column along the extension direction of the text content of the nth text column, or the coordinates of M points can be collected from the left or right area of ​​the nth text column along the extension direction of the text content of the nth text column.

[0121] In some embodiments, the M points collected from the area containing the nth text line or the nth text column can be equidistant or non-equidistant, and this disclosure does not limit this.

[0122] For example, when displaying the first text image via an electronic device, M points can be determined on the nth text unit by manual annotation, and the coordinates of each of the M points included in the nth text unit can be determined based on the first coordinate system provided above, and the coordinates of each determined point can be used as the first coordinates included in the nth text unit.

[0123] For example, for the nth text unit in the first text image, a pixel coordinate can be obtained from a character, and the pixel coordinates obtained from the multiple characters included in the nth text unit can be used as the first coordinates included in the nth text unit.

[0124] For example, a text unit can also be defined by a first text box based on its position in the first text image. In this case, the coordinates of the first text box in the first text image can reflect, to some extent, the positional area of ​​the text unit defined by the first text box in the first text image, as well as the shape of the defined text unit. Therefore, N text units in the first text image can also be defined by N first text boxes, and the coordinates of M points can be obtained from the border of the first text box used to define the nth text unit. The coordinates of the M points obtained from the border of the first text box defining the nth text unit are used as the first coordinates of the corresponding text unit.

[0125] The above are merely examples illustrating several methods for obtaining the first coordinate, and this disclosure does not limit these methods.

[0126] The implementation methods for generating a first text box on each text unit in the first text image include:

[0127] The first text image is input into the text detection network. The text detection network detects the edge regions of each text unit and generates a first text box based on the detected edge regions of each text unit to define the corresponding text unit.

[0128] Alternatively, a first text box can be drawn manually on each text unit.

[0129] In this embodiment of the disclosure, a first text box may also be generated on each text unit of the first text image in other ways, and this embodiment of the disclosure does not limit this.

[0130] It should be noted that if the number of first coordinates collected from each text unit is small, the outer contour features of the text unit may not be reflected. If the number of first coordinates collected from each text unit is large, it will increase the subsequent calculation load and the number of coordinate points that need to be corrected, which may result in a waste of computing resources.

[0131] Therefore, in one embodiment, the number of first coordinates collected from the corresponding text unit can be determined according to the degree of deformation of each text unit. When the degree of deformation of the text unit is small, a smaller number of first coordinates can be collected, and when the degree of deformation of the text unit is large, a larger number of first coordinates can be collected. This not only ensures the correction effect of the text unit, but also improves the correction efficiency.

[0132] For example, for Figure 2 The text image shown can extract 20-25 first coordinates from each line of text.

[0133] In some embodiments, since the line connecting all the first coordinates included in each text unit can reflect the shape of the outer contour of the corresponding text unit, after inputting the first coordinates of all text units obtained from the first text image into the encoder, the encoder can obtain the nth row of the first feature vector according to the outer contour of the nth text unit indicated by the line connecting all the first coordinates included in the nth text unit.

[0134] For example, for Figure 2 The left half of the middle Figure 3 The left half of the middle and Figure 5 The left half of the image shows a first text image, where the text unit is a text line. Since the text lines in the first text image are all upward-convex arcs, after acquiring multiple first coordinates from any text line, the line connecting these multiple first coordinates is also an upward-curving broken line or arc. Therefore, after inputting the first coordinates of the nth text line obtained from the first text image into the encoder, the encoder can know the outer contour shape of the nth text line based on the line connecting the first coordinates included in the nth text line, and thus obtain the nth row of the first feature vector used to indicate the outer contour shape of the nth text line.

[0135] For example, for Figure 4 The left half of the middle and Figure 6The first text image shown in the left half of the image has text units as text lines. The line connecting all the first coordinates of any text line is a downward-sloping straight line or a broken line. In this case, after the first coordinates of the nth text line obtained from the first text image are input into the encoder, the encoder can know that the outer contour of the nth text line is a downward-sloping straight line based on the line connecting the first coordinates of the nth text line, and thus obtain the nth line of the first feature vector used to indicate the shape of the outer contour of the nth text line.

[0136] The encoder described above can be an encoder that includes multiple linear layers and multiple activation layers. The number of linear layers and activation layers can be designed as needed, and this embodiment does not limit this.

[0137] For example, the encoder described above may include three linear layers and three corresponding activation layers.

[0138] After inputting the first coordinates of N text units acquired from the first text image into the encoder, the encoder extracts the outer contour features of each text unit in the first text image through a first linear layer and a first activation layer to obtain a third feature vector. Then, the third feature vector is input into the second linear layer and the second activation layer of the encoder for further feature extraction to obtain a fourth feature vector. Finally, the fourth feature vector is input into the third linear layer and the third activation layer of the encoder for further feature extraction to obtain a first feature vector.

[0139] In one embodiment, the method further includes:

[0140] Based on the text distribution of each text unit in the first text image, obtain the end coordinates of each text unit;

[0141] Based on the text distribution of each text unit, one or more intermediate coordinates are collected between any two end coordinates.

[0142] In this embodiment of the disclosure, when acquiring the first coordinates of each text unit of the first text image, the end coordinates of each text unit can be acquired first to determine the start and end positions of each text unit in the first text image. Then, the coordinates of one or more points between the two end coordinates of each text unit are acquired as the middle coordinates of the text unit.

[0143] For example, if the text unit is a text line, when collecting the end coordinates of each text line, one or more points can be determined on the left side of the first character and the right side of the last character of each text line, and the coordinates of the one or more points determined on the left side of the first character and the right side of the last character are used as the end coordinates of the text line. Then, coordinates of one or more points collected between the two end coordinates of each text line are used as the middle coordinates of the text line.

[0144] For example, with respect to Figure 2 , for the third text line, one or more coordinates can be collected from the left side of the character "二" as the left end coordinate of the third text line. One or more coordinates are collected from the right side of the last character "1" of the third text line as the right end coordinate of the third text line, and the collected left end coordinate and right end coordinate are used as the end coordinates of the third text line. Then, multiple coordinates can be collected from the area where the characters "construction conditions………………" are located, which are used as the middle coordinates of the third text line.

[0145] For example, if the text unit is a text column, when collecting the end coordinates of each text column, one or more points can be determined above the first character and below the last character of each text column, and the coordinates of the one or more points determined above the first character and below the last character are used as the end coordinates of the text column. Then, coordinates of one or more points collected between the two end coordinates of each text column are used as the middle coordinates of the text column.

[0146] For example, when collecting the middle coordinates of each text unit, the number of collected middle coordinates can be determined according to the deformation degree of each text unit.

[0147] In some embodiments, when the first text image is Figure 2 , the left half of Figure 3 , the left half of Figure 5 , which is a curved text image as shown by the left half, the number of middle coordinates collected from each text unit can be determined according to the bending degree of the text unit.

[0148] When the bending degree of the text unit is relatively large, a larger number of middle coordinates can be collected. For example, the number of middle coordinates collected from each text unit can be any value from 20 to 40. When the bending degree of each text unit is relatively small, a smaller number of middle coordinates can be collected. For example, the number of middle coordinates collected from each text unit is any value from 10 to 20.

[0149] In some embodiments, the degree of curvature can be divided by the curvature corresponding to the curve formed by connecting the first coordinates. The curvature is located in different curvature intervals and can correspond to different numbers of intermediate coordinates.

[0150] When the first text image is Figure 4 The left half or Figure 6 When the left half of the image shows a tilted text image, the number of intermediate coordinates collected can be determined based on the tilt degree of the nth text unit.

[0151] The degree of tilt can be distinguished by the angle between the straight line connecting the first coordinates and the specified line. Different angle ranges correspond to different numbers of intermediate coordinates.

[0152] When the text unit has a large degree of slant, a larger number of intermediate coordinates can be collected; for example, the number of intermediate coordinates collected from each text unit can be any value between 20 and 40. When the text unit has a small degree of slant, a smaller number of intermediate coordinates can be collected; for example, the number of intermediate coordinates collected from each text unit can be any value between 10 and 20.

[0153] In one embodiment, based on the text distribution of each text unit, one or more intermediate coordinates are acquired between any two end coordinates, including:

[0154] The first text image is segmented using N first text boxes, where each first text box is used to define a text unit.

[0155] Determine the end coordinates of the nth text unit based on the vertex position of the nth first text box;

[0156] Collect one or more intermediate coordinates on the border of the nth first text box.

[0157] In this embodiment of the disclosure, a text unit can be defined by one of the N first text boxes. In this case, the vertex coordinates of each first text box can be used as the end coordinates of the corresponding text unit, and one or more coordinates obtained from the top and bottom borders of each first text box can be used as the middle coordinates of the corresponding text unit.

[0158] For example, refer to Figures 2 to 6 In the left half of the text, each line of text is enclosed by a first text box. At this time, the coordinates of the top left, bottom left, top right and bottom right corners of each first text box can be used as the end coordinates of the text line enclosed by the first text box, and the coordinates of multiple points collected from the top and bottom borders of the first text box can be used as the middle coordinates of the text line enclosed by the first text box.

[0159] In one embodiment, the method further includes:

[0160] Determine the number of intermediate coordinates set on the border of the nth first text box based on the curvature and / or slope of the outer contour of the nth first text box.

[0161] In this embodiment of the disclosure, if the text unit in the first text image is a text line, when collecting the intermediate coordinates of the nth text unit from the border of the nth first text box, the number of intermediate coordinates collected from the border of the nth first text box can be determined according to the curvature of the upper border and / or lower border of the nth first text box, or the number of intermediate coordinates collected from the border of the nth first text box can be determined according to the slope of the upper border and / or lower border of the nth first text box.

[0162] For example, when the curvature of the top and / or bottom borders of the nth first text box is large, a larger number of intermediate coordinates can be collected from the top and / or bottom borders of the nth first text box; when the curvature of the top and / or bottom borders of the nth first text box is small, a smaller number of intermediate coordinates can be collected from the top and / or bottom borders of the nth first text box.

[0163] For example, when the slope of the top and / or bottom border of the nth first text box is large, a larger number of intermediate coordinates can be collected from the top and / or bottom border of the nth first text box; when the slope of the top and / or bottom border of the nth first text box is small, a smaller number of intermediate coordinates can be collected from the top and / or bottom border of the nth first text box.

[0164] When the text unit in the first text image is a text column, the middle coordinates of each text column can be collected from the left border and / or right border of the first text box. The implementation method of collecting the middle coordinates of each text column from the left border and / or right border of the first text box can refer to the above implementation method of collecting the middle coordinates of each text line from the top border and / or bottom border of the first text box. This embodiment of the present disclosure will not be repeated here.

[0165] When the text unit is a text line, the curvature of the outer contour of the nth text line can be determined based on the distance along the Y-axis between the coordinates of the top left corner of the first text box and the coordinates of the top border with the maximum or minimum ordinate value. Specifically, the greater the distance along the Y-axis between the top left corner of the first text box and the coordinates of the top border with the maximum or minimum ordinate value, the greater the curvature of the first text box; conversely, the smaller the distance along the Y-axis between the top left corner of the first text box and the coordinates of the top border with the maximum or minimum ordinate value, the less curvature of the first text box.

[0166] The slope of the outer contour of the nth text unit can be calculated using the coordinates of the two ends located at the top and / or bottom edges of the first text box.

[0167] When the text unit is a text column, the curvature of the nth text column can be determined based on the distance along the X-axis between the top-left corner of the first text box and the point with the maximum or minimum horizontal coordinate value on the left border. Specifically, the greater the distance along the X-axis between the top-left corner of the first text box and the point with the maximum or minimum horizontal coordinate value on the left border, the greater the curvature of the first text box; conversely, the smaller the distance, the less curvature.

[0168] The slope of the outer contour of the nth text unit can be calculated using the coordinates of the two ends located at the left and / or right ends of the first text box.

[0169] In this embodiment of the disclosure, the curvature and / or slope of the first text box can also be determined in other ways, and this embodiment of the disclosure does not limit this.

[0170] In one embodiment, S101 further includes:

[0171] Based on the multiple first coordinates included in each of the N text units, a first matrix is ​​obtained; wherein the first matrix has N rows and M columns, one row of the first matrix corresponds to one text unit, and M is the number of first coordinates included in one text unit;

[0172] Interpolation is performed on each row of the first matrix to obtain the first eigenvector.

[0173] In some embodiments, before inputting the first coordinates of the N text units of the first text image into the encoder for interpolation processing, a first matrix can be generated based on the first coordinates of the N text units of the first text image. Each element of the first matrix corresponds to a first coordinate in the first text image. When a text line in the first text image is considered a text unit, the M first coordinates of the nth text line can be used as the nth row of the first matrix, resulting in a first matrix with N rows and M columns. When a text column in the first text image is considered a text unit, the M first coordinates of the nth column can be used as the nth row of the first matrix, resulting in a first matrix with N rows and M columns. After obtaining the first matrix, it is input into the encoder so that the encoder inserts one or more second coordinates between two adjacent first coordinates in a row of the first matrix, resulting in a first feature vector.

[0174] In some embodiments, the number of second coordinates inserted in each text line of the first matrix can be set as needed, and this disclosure does not limit this.

[0175] For example, the number of second coordinates to be inserted between adjacent elements can be determined based on the distance between two adjacent first coordinates in the same row of the first matrix. When the distance between two adjacent first coordinates in the same row is large, a larger number of second coordinates can be inserted between them. For example, 3 to 5 second coordinates can be inserted between two adjacent first coordinates in the same row. When the distance between two adjacent first coordinates in the same row is small, a smaller number of second coordinates can be inserted between them. For example, 1 to 2 second coordinates can be inserted between two adjacent first coordinates in the same row.

[0176] In one embodiment, interpolation is performed on each row of the first matrix to obtain a first feature vector, including:

[0177] The first matrix is ​​normalized to obtain the second matrix;

[0178] Interpolation is performed on each row of the second matrix to obtain the first eigenvector.

[0179] In this embodiment of the disclosure, before interpolating the first matrix into the encoder, each element of the first matrix can be normalized to obtain a second matrix, and then the second matrix can be input into the encoder for interpolation to obtain the first feature vector output by the encoder.

[0180] In some embodiments, the implementation of interpolation processing of the second matrix by the encoder can refer to the implementation of interpolation processing of the first matrix by the encoder, and will not be described again in this disclosure.

[0181] The normalization process for the first matrix includes:

[0182] Obtain the maximum x-coordinate from the first matrix. max and minimum value x min and the maximum value of the ordinate y max and minimum value y min ;

[0183] The normalized coordinate value of the x-coordinate in the i-th first coordinate is calculated using the following formula (1);

[0184]

[0185] Where, x i x is the normalized x-coordinate value of the i-th first coordinate;i x is the x-coordinate of the i-th first coordinate before normalization; max x is the x-coordinate of the largest coordinate value included in the first matrix; min It is the coordinate value of the x-coordinate with the smallest coordinate value included in the first matrix.

[0186] The normalized coordinate value of the ordinate in the i-th first coordinate is calculated using the following formula (2);

[0187]

[0188] Among them, y i Let y be the normalized ordinate value of the i-th first coordinate. i Let y be the ordinate value of the i-th first coordinate before normalization. max The y-coordinate is the coordinate of the largest ordinate in the first matrix. min It is the coordinate value of the ordinate with the smallest coordinate value included in the first matrix.

[0189] In one embodiment, the first feature vector and the second feature vector are fused to obtain a third feature vector, including:

[0190] The first and second eigenvectors are added together to obtain the third eigenvector.

[0191] In this embodiment of the disclosure, addition can be performed directly on the first feature vector and the second feature vector, or a first weight coefficient can be assigned to the first feature vector and a second weight coefficient can be assigned to the second feature vector according to the importance of the first feature vector and the second feature vector in correcting the first coordinate, and a weighted sum of the first feature vector and the second feature vector can be calculated based on the first weight coefficient and the second weight coefficient, and the weighted sum of the first feature vector and the second feature vector can be used as the third feature vector.

[0192] For example, considering that different deformation modes of the first text image may require different correction methods, and that the first feature vector can reflect the deformation of text units, it is evident that the contribution of the first feature vector to coordinate correction is greater than that of the second feature vector. Therefore, a larger first weight coefficient can be assigned to the first feature vector, and a smaller weight coefficient can be assigned to the second feature vector.

[0193] For example, the first weight coefficient assigned to the first feature vector is any value from 0.6 to 0.8.

[0194] In one embodiment, correcting the first text image based on a third feature vector to obtain a second text image includes:

[0195] Perform at least one convolution and at least one pooling operation on the third feature vector to obtain the second coordinates after correcting the first coordinates;

[0196] A second text image is generated based on the text content and second coordinates of the first text image.

[0197] In some embodiments, the third feature vector can be input into the decoder, and the third feature vector can be subjected to at least one convolutional process and at least one pooling process to obtain the second coordinates after the first coordinates have been corrected.

[0198] In some cases, the decoder may include multiple linear layers and multiple activation layers. The number of linear layers and activation layers included in the decoder can be designed as needed, and this disclosure does not limit this.

[0199] For example, the decoder in the disclosed embodiment may include three linear layers and two activation layers. The third feature vector is input into the first linear layer and the first activation layer of the decoder for dimensionality reduction processing to obtain a sixth feature vector. The sixth feature vector is input into the second linear layer and the second activation layer of the decoder for dimensionality reduction processing again to obtain a seventh feature vector. The seventh feature vector is input into the third linear layer of the decoder to obtain the second coordinates output by the decoder.

[0200] In one embodiment, generating a second text image based on the text content and second coordinates of a first text image includes:

[0201] Generate the nth text unit of the second text image based on the text content and second coordinates of the nth text unit.

[0202] For example, the second text image has the same size as the first text image. After obtaining the second coordinates of the nth text unit, the second coordinates of the nth text unit are determined in a first region of the second text image. The text content identified from the nth text unit of the first text image is written into the first region to obtain the nth text unit of the second text image.

[0203] For example, after correcting the end coordinates of the nth text unit to obtain the second coordinates, the second coordinates can be used as vertices to generate the nth second text box in the second text image. Then, the text content of the nth text unit is written into the nth text box of the second text image.

[0204] For example, when a text line in the first text image is a text unit, the second text box generated based on the second coordinates can be... Figures 2 to 6 The text box shown in the right half of the image.

[0205] In some embodiments, reference Figure 7Before correcting the first text image, it is also possible to... Figure 7 As shown, a text correction model including an encoder, an LSTM network, and a decoder is trained and tested, enabling the encoder to interpolate the first matrix to obtain the first feature vector, enabling the LSTM network to correct the first feature vector based on the features of different columns in the first feature vector to obtain the second feature vector, and enabling the decoder to obtain the second coordinates after correcting the first coordinates based on the third feature vector.

[0206] For example, Figure 8 The diagram illustrates a flowchart for training a text correction model. Figure 8 As can be seen, the training process for the text correction model includes:

[0207] Obtain the training set, which includes deformed text images and normal text images corresponding to each deformed text image;

[0208] The third coordinate is acquired from the deformed text image, and the fourth coordinate corresponding to the third coordinate is acquired from the normal text image;

[0209] Based on the third coordinates acquired from each deformed text image, a third matrix is ​​generated. This third matrix is ​​then input into a text correction model, where an encoder interpolates each row to obtain the eighth feature vector. Next, the first feature vector is input into an LSTM network, which corrects the eighth feature vector using features from different columns, outputting a ninth feature vector. After obtaining the ninth feature vector, the eighth and ninth feature vectors are fused, and the resulting tenth feature vector is input into a decoder. The decoder decodes the tenth feature vector to obtain the fifth coordinate, which has been corrected based on the third coordinates.

[0210] The loss value is calculated based on the difference between the fifth coordinate and the corresponding fourth coordinate, and the parameters of the encoder, LSTM network and decoder of the text correction model are corrected based on the loss value.

[0211] During the training process described above, the encoder in the text correction model can learn to interpolate the first matrix composed of the first coordinates of the deformed text image, the LSTM network can learn to correct the first feature vector according to the features of different columns of the first feature vector to obtain the second feature vector, and the decoder can learn to decode the third feature vector to obtain the second coordinate after correcting the first coordinate.

[0212] After training the text correction model, its training performance can be tested. The testing process for the text correction model can be found by referring to... Figure 9 The embodiments disclosed herein will not be described in detail here.

[0213] refer to Figure 10 , Figure 10 A flowchart illustrating another text image processing method provided in this disclosure embodiment is shown below. Figure 10 As shown, the text image processing method includes:

[0214] Text detection: The first text image is input into the DBNet text detection network. The DBNet text detection network identifies the edge positions of each text line in the first text image and generates a first text box based on the identified edge positions of each text line. Each first text box is used to define a single text line.

[0215] Collect the coordinates of M points on the border of each first text box, and use the coordinates of the M points collected from the border of each first text box as the first coordinates of the text line enclosed by that text line.

[0216] Coordinate serialization and normalization: Text coordinate serialization refers to the process where the DBNet text detection network generates N first text boxes (meaning the first text image contains N text lines) after detecting the first text image, with each first text box described by M points (x, y). At this point, a tensor X0 with dimensions [N, M, 2] is obtained. This tensor indicates that there are N first text boxes in the first text image, and each first text box has M coordinate points. Then, the first coordinates obtained based on the first coordinate of the top-left corner of each first text box are sorted. The purpose of this sorting is to form a sequence of first coordinates from the N first text boxes, resulting in the first coordinate sequence.

[0217] Then, the x and y coordinate values ​​of the first coordinate included in the first coordinate sequence are normalized. The normalization method is Min-Max normalization. The formula used for normalization can be referred to the above formulas (1) and (2). After normalizing the first coordinate sequence, the first coordinate sequence can be transformed into a two-dimensional matrix X1 with dimensions [N, M*2].

[0218] The sequence coordinates are encoded into sparse feature vectors as follows: First, the first sequence coordinates X1 are fed into the encoder, which consists of three linear layers and three ReLU activation layers. The first coordinate sequence X1 is input into the first linear layer to obtain the fourth feature vector X2. The fourth feature vector X2 is then fed into the second linear layer after passing through the first activation layer, and then activated by the second activation layer to obtain the fifth feature vector X3. The fifth feature vector X3 is then fed into the third linear layer, and after passing through the third linear layer and the third activation layer, the first feature vector f1 is obtained. The dimension of the first feature vector f1 is [N, 256]. Then, the first feature vector f1 is input into the LSTM network to learn the dependencies between the text lines, resulting in the second feature vector f2, which also has a dimension of [N, 256].

[0219] Feature fusion and coordinate decoding: The first feature vector f1 and the second feature vector f2 are fused to obtain the third feature vector f, i.e., f = f1 + f2. f is then fed into the decoder, which consists of three linear layers and two ReLU activation layers. The third feature vector f passes through the first linear layer to obtain the sixth feature vector H1. The sixth feature vector H1 passes through the first activation layer and is then fed into the second linear layer to obtain the seventh feature vector H2. The seventh feature vector H2 is then fed into the third linear layer to obtain the corrected second coordinates. The matrix dimension of the second coordinates is [N, 4], where N represents the number of first text boxes, and 4 represents the four coordinate values ​​of the first text box, namely the coordinates of the top-left, bottom-left, top-right, and bottom-right corners.

[0220] like Figure 11 As shown, this disclosure provides a text image processing apparatus, including:

[0221] The module 1101 is used to obtain a first feature vector based on multiple first coordinates of each text unit in N text units; the N text units are in a first text image; the first feature vector has N rows and X columns, and one row in the first feature vector corresponds to one text unit; N is a positive integer, and X is an integer greater than or equal to 2;

[0222] The first correction module 1102 is used to correct the first feature vector according to the features of different columns of the first feature vector to obtain a second feature vector, wherein the second feature vector has N rows and X columns, and one row in the second feature vector corresponds to one text unit;

[0223] The fusion module 1103 is used to fuse the first feature vector and the second feature vector to obtain the third feature vector, which has N rows and Y columns; wherein, one row of the third feature vector is used for the position correction of a text unit.

[0224] The second correction module 1104 is used to correct the first text image based on the third feature vector to obtain the second text image.

[0225] Optionally, a text unit includes M first coordinates;

[0226] Module 1101 is obtained and used for:

[0227] Extract the outer contour corresponding to the first coordinate line of the nth text unit to obtain the nth row of the first feature vector; n is a positive integer less than or equal to N; N is the total number of rows of text units in the first text image.

[0228] Optionally, a plurality of first coordinates of a text cell include: end coordinates and intermediate coordinates located between the end coordinates, wherein the end coordinates are located at both ends of the text cell;

[0229] The device also includes:

[0230] The acquisition module is used to acquire the end coordinates of each text unit based on the text distribution of each text unit in the first text image;

[0231] The first setting module is used to collect one or more intermediate coordinates between any two end coordinates based on the text distribution of each text unit.

[0232] Optionally, the acquisition module is used for:

[0233] The first text image is segmented using N first text boxes, where each first text box is used to define a text unit.

[0234] Determine the end coordinates of the nth text unit based on the vertex position of the nth first text box;

[0235] Collect one or more intermediate coordinates on the border of the nth first text box.

[0236] Optionally, the device further includes:

[0237] The determination module is used to determine the number of intermediate coordinates collected on the border of the nth first text box based on the curvature and / or slope of the outer contour of the nth first text box.

[0238] Optionally, the fusion module 1103 is used for:

[0239] The first and second eigenvectors are added together to obtain the third eigenvector.

[0240] Optionally, the module is also used for:

[0241] Based on the multiple first coordinates included in each of the N text units, a first matrix is ​​obtained, wherein the first matrix has N rows and M columns, one row of the first matrix corresponds to one text unit, and M is the number of first coordinates included in one text unit;

[0242] Interpolation is performed on each row of the first matrix to obtain the first eigenvector.

[0243] Optionally, module 1101 is also used for:

[0244] The first matrix is ​​normalized to obtain the second matrix;

[0245] Interpolation is performed on each row of the second matrix to obtain the first eigenvector.

[0246] Optionally, the second calibration module 1104 is used for:

[0247] Perform at least one convolution and at least one pooling operation on the third feature vector to obtain the second coordinates after correcting the first coordinates;

[0248] A second text image is generated based on the text content and second coordinates of the first text image.

[0249] refer to Figure 12 Embodiments of this disclosure also provide an electronic device, including:

[0250] processor;

[0251] Memory used to store processor-executable instructions;

[0252] The processor is configured to implement the text image processing method of any embodiment of this disclosure when running executable instructions.

[0253] The electronic device may be at least the aforementioned server and / or terminal device.

[0254] The memory can include various types of storage media, which are non-transitory computer storage media that can continue to store information after the communication device loses power.

[0255] The processor can connect to memory via a bus or similar means to read executable programs stored in memory, for example, to implement... Figure 1 and / or Figure 10 The method shown.

[0256] This disclosure also provides a computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the text image processing method of any embodiment of this disclosure. For example, implementing... Figure 1And / or the method shown in 10.

[0257] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0258] This disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to, when executing the executable instructions, implement the table processing method provided by any of the foregoing technical solutions, for example, ... Figure 1 and / or Figure 10 The method shown.

[0259] like Figure 10 As shown, one embodiment of this disclosure illustrates the structure of an electronic device. For example, the electronic device 1200 can be provided as a network-side device. (Refer to...) Figure 10 Electronic device 1200 includes a processing component 1222, which further includes one or more processors, and memory resources represented by memory 1232 for storing instructions executable by the processing component 1222, such as application programs. The application programs stored in memory 1232 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1222 is configured to execute a program to perform any of the methods described above applied to the access device, such as... Figure 1 and / or Figure 10 The method shown.

[0260] Electronic device 1200 may also include a power supply component 1226 configured to perform power management of electronic device 1200, a wired or wireless network interface 1250 configured to connect electronic device 1200 to a network, and an input / output (I / O) interface 1258. Electronic device 1200 may operate on an operating system stored in memory 1232, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0261] In an exemplary embodiment, this disclosure provides a computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the text image processing method provided by any of the foregoing technical solutions.

[0262] Specifically, the computer-readable storage medium may be a non-transitory computer-readable storage medium.

[0263] For example, a memory 1232 including an executable program may be used to execute the above instructions by a processing component 1222 of an electronic device 1200 to perform the above method.

[0264] Non-transitory computer-readable storage media can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc.

[0265] Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the embodiments of the present disclosure that follow the general principles of the embodiments of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the embodiments of the present disclosure are indicated by the following claims.

[0266] It should be understood that the embodiments disclosed herein are not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from their scope. The scope of the embodiments disclosed herein is limited only by the appended claims.

Claims

1. A text image processing method, characterized in that, The method includes: A first feature vector is obtained based on multiple first coordinates of each of the N text units; the N text units are in the first text image; the first feature vector has N rows and X columns, and one row of the first feature vector corresponds to one text unit; N is a positive integer, and X is an integer greater than or equal to 2; The first feature vector is corrected based on the features of different columns of the first feature vector to obtain a second feature vector; the second feature vector has N rows and X columns, wherein one row of the second feature vector corresponds to one text unit; The first feature vector and the second feature vector are fused to obtain a third feature vector, which has N rows and Y columns; wherein, each row of the third feature vector is used for the position correction of one text unit; The first text image is corrected based on the third feature vector to obtain the second text image.

2. The method according to claim 1, characterized in that, One of the text units includes M of the first coordinates; The step of obtaining the first feature vector based on multiple first coordinates of each of the N text units includes: Extract the outer contour corresponding to the first coordinate line of the nth text unit to obtain the nth row of the first feature vector, where n is a positive integer less than or equal to N.

3. The method according to claim 1, characterized in that, A plurality of first coordinates of a text unit includes: end coordinates and intermediate coordinates located between the end coordinates, the end coordinates being located at both ends of the text unit; The method further includes: Based on the text distribution of each text unit in the first text image, obtain the end coordinates of each text unit; Based on the text distribution of each text unit, one or more intermediate coordinates are collected between any two end coordinates.

4. The method according to claim 3, characterized in that, The step of acquiring one or more intermediate coordinates between any two end coordinates based on the text distribution of each text unit includes: The first text image is segmented by N first text boxes, wherein one first text box is used to define one text unit; Based on the position of the vertex of the nth first text box, determine the end coordinates of the nth text unit; One or more of the intermediate coordinates are collected on the border of the nth first text box.

5. The method according to claim 4, characterized in that, The method further includes: The number of intermediate coordinates collected on the border of the nth first text box is determined based on the curvature and / or slope of the outer contour of the nth first text box.

6. The method according to any one of claims 1-5, characterized in that, The process of fusing the first feature vector and the second feature vector to obtain the third feature vector includes: The first feature vector and the second feature vector are added together to obtain the third feature vector.

7. The method according to any one of claims 1-5, characterized in that, The step of obtaining the first feature vector based on multiple first coordinates of each of the N text units includes: Based on the plurality of first coordinates included in each of the N text units, a first matrix is ​​obtained, wherein the first matrix has N rows and M columns, one row of the first matrix corresponds to one text unit, and M is the number of first coordinates included in one text unit; Interpolation is performed on each row of the first matrix to obtain the first feature vector.

8. The method according to claim 7, characterized in that, The step of interpolating each row of the first matrix to obtain the first feature vector includes: The first matrix is ​​normalized to obtain the second matrix; Interpolation is performed on each row of the second matrix to obtain the first feature vector.

9. The method according to any one of claims 1-5, characterized in that, The step of correcting the first text image based on the third feature vector to obtain the second text image includes: Perform at least one convolution and at least one pooling operation on the third feature vector to obtain the second coordinates after correcting the first coordinates; The second text image is generated based on the text content of the first text image and the second coordinates.

10. A text image processing apparatus, characterized in that, The device includes: The module is configured to obtain a first feature vector based on multiple first coordinates of each of the N text units; the N text units are in a first text image; the first feature vector has N rows and X columns, and one row of the first feature vector corresponds to one of the text units; N is a positive integer, and X is an integer greater than or equal to 2; The first correction module is used to correct the first feature vector according to the features of different columns of the first feature vector to obtain a second feature vector; wherein the second feature vector has N rows and X columns, and one row of the second feature vector corresponds to one text unit; A fusion module is used to fuse the first feature vector and the second feature vector to obtain a third feature vector, the third feature vector having N rows and Y columns; wherein, one row of the third feature vector is used for the position correction of one text unit; The second correction module is used to correct the first text image based on the third feature vector to obtain the second text image.

Images