Table extraction method, device and touch display device

Abstract

The present invention provides a table extraction method, device and touch display device, which relate to the field of image recognition technology. The present invention receives an operation instruction to take a screenshot of the content displayed on the screen of the touch display device, takes a screenshot of the content displayed on the screen according to the operation instruction, obtains a screenshot image, detects the screenshot image, obtains a second table corresponding to a first table, identifies characters in the screenshot image, establishes a matching relationship between the second table and the characters, and saves the second table, the characters and the matching relationship. When the user wants to save the content displayed on the screen, the screenshot can be taken by triggering the operation instruction to take a screenshot of the content displayed on the screen, and then the second table corresponding to the first table in the screenshot image and the characters in the second table are automatically extracted, and the second table and the characters are saved as a target document. The user can subsequently edit the content displayed on the screen again based on the target document, thereby improving the user's operating experience.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a table extraction method, device and touch display device. Background Art

[0002] Touch display devices usually use a touch-sensitive display screen as a carrier, and use multi-point infrared, optical and other touch methods for human-computer interaction to achieve operations such as writing on the touch display screen. Touch display screens are usually used in scenarios such as meetings and teaching.

[0003] Currently, users can draw tables, Chinese, English, numbers and other content on the touch display screen. After drawing is completed, if the user needs to save the drawn content, the content drawn on the screen can be saved as a picture by taking a screenshot.

[0004] However, the content drawn on the screen is saved in the form of a picture, and the user cannot edit the drawn content again after opening it again. Summary of the invention

[0005] The present invention provides a table extraction method, device and touch display device to solve the problem that the existing contents drawn on the screen are saved in the form of pictures and the user cannot edit the drawn contents again after opening the picture again.

[0006] In order to solve the above problem, the present invention discloses a table extraction method, which is applied to a touch display device, and the method comprises:

[0007] Receiving an operation instruction to take a screenshot of the content displayed on the screen of the touch display device;

[0008] Taking a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen;

[0009] Detecting the screenshot to obtain a second table corresponding to the first table;

[0010] Recognize characters in the screenshot, and establish a matching relationship between the second table and the characters;

[0011] The second table, the characters and the matching relationship are saved to obtain a target document.

[0012] Optionally, the step of detecting the screenshot to obtain a second table corresponding to the first table includes:

[0013] Identifying candidate curve segments corresponding to the first table in the screenshot;

[0014] The candidate curve segments are processed to generate a second table corresponding to the first table.

[0015] Optionally, the step of identifying candidate curve segments corresponding to the first table in the screenshot includes:

[0016] Processing the screenshot to obtain a binary image;

[0017] According to a first preset order, the first mask pattern is used to sequentially traverse the neighboring pixel points of each candidate pixel point in the binary image row by row;

[0018] Determine neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtain a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction;

[0019] According to a second preset order, the second mask pattern is used to sequentially traverse the neighboring pixel points of each candidate pixel point in the binary image column by column;

[0020] Neighboring pixels having the same set value as the candidate pixel are determined as similar pixels, and a plurality of second candidate curve segments corresponding to the first table are obtained; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

[0021] Optionally, the step of processing the candidate curve segments to generate a second table corresponding to the first table includes:

[0022] Fitting each of the first candidate curve segments and each of the second candidate curve segments respectively to obtain fitted first broken line segments and second broken line segments;

[0023] Merging the plurality of first fold line segments and the plurality of second fold line segments according to a first preset condition to obtain corresponding first table lines and second table lines;

[0024] A second table corresponding to the first table is generated according to the first table line and the second table line.

[0025] Optionally, before the step of fitting each of the first candidate curve segments and each of the second candidate curve segments respectively, the method further includes:

[0026] The first candidate curve segment and the second candidate curve segment whose length is less than the first preset length are screened out.

[0027] Optionally, the step of merging the plurality of first fold line segments and the plurality of second fold line segments according to the first preset condition to obtain corresponding first table lines and second table lines includes:

[0028] For any two first polyline segments, if the endpoint of the first straight line segment in the first candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the first polyline segment to be merged is located, merge the first candidate polyline segment and the first polyline segment to be merged;

[0029] For any two second polyline segments, if the endpoint of the first straight line segment in the second candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the second polyline segment to be merged is located, merge the second candidate polyline segment and the second polyline segment to be merged;

[0030] The first preset range is determined by a first preset angle, a second preset angle, and a second preset length; when the first angle is smaller than the first preset angle, the second angle is smaller than the second preset angle, and the first length is smaller than the second preset length, the endpoint of the first straight line segment is located within the first preset range where the endpoint of the second straight line segment is located;

[0031] A first line segment is formed between the endpoint of the first straight line segment and the endpoint of the second straight line segment, the first angle is the angle between the first line segment and the extension line of the second straight line segment, the second angle is the angle between the extension line of the first straight line segment and the extension line of the second straight line segment, and the first length is the length of the perpendicular segment between the endpoint of the first straight line segment and the extension line of the second straight line segment.

[0032] Optionally, the step of generating a second table corresponding to the first table according to the first table line and the second table line includes:

[0033] sorting the plurality of second table lines according to a third preset order;

[0034] According to the sorting result, a second table line satisfying a second preset condition is selected from the second table lines as the first boundary line; the second preset condition is that there are two first table lines within a second preset range of two end points of the second table line, and the length difference between the two first table lines is less than a third preset length;

[0035] From the remaining second table lines, a second table line that meets a third preset condition is selected as the second boundary line, and the two first table lines are determined as the third boundary line and the fourth boundary line; the third preset condition is that the two endpoints of the second table line are respectively located within a third preset range where the first endpoints of the two first table lines are located;

[0036] The remaining first table lines and second table lines are screened according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain a table middle line to generate a second table.

[0037] Optionally, the step of screening the remaining first table lines and second table lines according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain the table middle line includes:

[0038] Selecting, from the remaining first table lines, a first table line that meets a fourth preset condition as a table middle line arranged along the first direction;

[0039] From the remaining second table lines, selecting a second table line that meets the fifth preset condition as a table middle line arranged along the second direction;

[0040] Among them, the fourth preset condition is that the distances between the two endpoints of the first table line and the first boundary line and the second boundary line are both less than the first preset distance; the fifth preset condition is that the distances between the two endpoints of the second table line and the third boundary line and the fourth boundary line are both less than the second preset distance.

[0041] Optionally, the step of establishing a matching relationship between the second table and the characters includes:

[0042] Obtaining first position information of each cell in the second table;

[0043] Obtaining second position information of each of the characters;

[0044] A matching relationship between the second table and the character is established according to the first position information and the second position information.

[0045] Optionally, after the step of saving the second table, the characters and the matching relationship to obtain the target document, the method further includes:

[0046] Generate a document title according to the storage time of the target document;

[0047] Alternatively, receive a document title input by the user;

[0048] The document title is added to the target document.

[0049] In order to solve the above problem, the present invention further discloses a table extraction device, which comprises:

[0050] An operation instruction receiving module, configured to receive an operation instruction to take a screenshot of the content displayed on the screen of the touch display device;

[0051] A screenshot module, configured to take a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen;

[0052] a table detection module, configured to detect the screenshot image to obtain a second table corresponding to the first table;

[0053] a character recognition module, configured to recognize characters in the screenshot and establish a matching relationship between the second table and the characters;

[0054] The saving module is configured to save the second table, the characters and the matching relationship to obtain a target document.

[0055] Optionally, the table detection module includes:

[0056] a candidate curve segment identification submodule, configured to identify candidate curve segments corresponding to the first table in the screenshot;

[0057] The second table generating submodule is configured to process the candidate curve segments to generate a second table corresponding to the first table.

[0058] Optionally, the candidate curve segment identification submodule includes:

[0059] A picture processing unit, configured to process the screenshot picture to obtain a binary image;

[0060] A first traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image row by row in sequence using a first mask pattern according to a first preset order;

[0061] A first candidate curve segment determination unit is configured to determine neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtain a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction;

[0062] A second traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image column by column in sequence using a second mask pattern according to a second preset order;

[0063] The second candidate curve segment determination unit is configured to determine neighboring pixel points having the same set value as the candidate pixel point as the same type of pixel points, and obtain multiple second candidate curve segments corresponding to the first table; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

[0064] Optionally, the second table generating submodule includes:

[0065] a candidate curve segment fitting unit, configured to fit each of the first candidate curve segments and each of the second candidate curve segments respectively to obtain fitted first broken line segments and second broken line segments;

[0066] A broken line segment merging unit, configured to merge the plurality of the first broken line segments and the plurality of the second broken line segments according to a first preset condition, respectively, to obtain corresponding first table lines and second table lines;

[0067] The second table generating unit is configured to generate a second table corresponding to the first table according to the first table line and the second table line.

[0068] Optionally, the table extraction device further includes:

[0069] The candidate curve segment screening unit is configured to screen out the first candidate curve segment and the second candidate curve segment whose lengths are less than a first preset length.

[0070] Optionally, the broken line segment merging unit includes:

[0071] A first merging subunit is configured to merge any two first polyline segments, if the endpoint of a first straight line segment in the first candidate polyline segment is located within a first preset range where the endpoint of a second straight line segment in the first polyline segment to be merged is located, the first candidate polyline segment and the first polyline segment to be merged are merged;

[0072] A second merging subunit is configured to merge any two second polyline segments, if the endpoint of the first straight line segment in the second candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the second polyline segment to be merged is located, the second candidate polyline segment and the second polyline segment to be merged are merged;

[0073] The first preset range is determined by a first preset angle, a second preset angle, and a second preset length; when the first angle is smaller than the first preset angle, the second angle is smaller than the second preset angle, and the first length is smaller than the second preset length, the endpoint of the first straight line segment is located within the first preset range where the endpoint of the second straight line segment is located;

[0074] A first line segment is formed between the endpoint of the first straight line segment and the endpoint of the second straight line segment, the first angle is the angle between the first line segment and the extension line of the second straight line segment, the second angle is the angle between the extension line of the first straight line segment and the extension line of the second straight line segment, and the first length is the length of the perpendicular segment between the endpoint of the first straight line segment and the extension line of the second straight line segment.

[0075] Optionally, the second table generating unit includes:

[0076] A second table line sorting subunit is configured to sort the plurality of second table lines according to a third preset order;

[0077] The first boundary line determination subunit is configured to select, from the second table lines according to the sorting result, a second table line that meets a second preset condition as the first boundary line; the second preset condition is that there are two first table lines within a second preset range of two end points of the second table line, and the length difference between the two first table lines is less than a third preset length;

[0078] The second boundary line determination subunit is configured to select a second table line that satisfies a third preset condition from the remaining second table lines as the second boundary line, and determine the two first table lines as the third boundary line and the fourth boundary line; the third preset condition is that the two end points of the second table line are respectively located within a third preset range where the first end points of the two first table lines are located;

[0079] The second table generating subunit is configured to screen the remaining first table lines and second table lines according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain a table middle line to generate a second table.

[0080] Optionally, the second table generating subunit is specifically configured as follows:

[0081] Selecting, from the remaining first table lines, a first table line that meets a fourth preset condition as a table middle line arranged along the first direction;

[0082] From the remaining second table lines, selecting a second table line that meets the fifth preset condition as a table middle line arranged along the second direction;

[0083] Among them, the fourth preset condition is that the distances between the two endpoints of the first table line and the first boundary line and the second boundary line are both less than the first preset distance; the fifth preset condition is that the distances between the two endpoints of the second table line and the third boundary line and the fourth boundary line are both less than the second preset distance.

[0084] Optionally, the character recognition module includes:

[0085] A first position information acquisition submodule, configured to acquire first position information of each cell in the second table;

[0086] A second position information acquisition submodule is configured to acquire the second position information of each of the characters;

[0087] The matching relationship establishing submodule is configured to establish a matching relationship between the second table and the character according to the first position information and the second position information.

[0088] Optionally, the table extraction device further includes:

[0089] A first title generation module is configured to generate a document title according to the storage time of the target document;

[0090] Alternatively, the second title generation module is configured to receive a document title input by a user;

[0091] The title adding module is configured to add the document title to the target document.

[0092] In order to solve the above problems, the present invention also discloses a touch display device, including a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above table extraction method when executed by the processor.

[0093] In order to solve the above problem, the present invention further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above table extraction method are implemented.

[0094] Compared with the prior art, the present invention has the following advantages:

[0095] By receiving an operation instruction to take a screenshot of the content displayed on the screen of the touch display device, taking a screenshot of the content displayed on the screen according to the operation instruction, obtaining a screenshot image, the screenshot image includes a first table displayed on the screen, detecting the screenshot image, obtaining a second table corresponding to the first table, identifying characters in the screenshot image, and establishing a matching relationship between the second table and the characters, saving the second table, the characters and the matching relationship to obtain a target document. When the user needs to save the content displayed on the screen, the operation instruction to take a screenshot of the content displayed on the screen can be triggered to take a screenshot of the content displayed on the screen, and then the second table corresponding to the first table in the screenshot image and the characters in the second table are automatically extracted, and the second table and the characters are saved as the target document. The user can subsequently edit the content displayed on the screen again based on the target document, thereby improving the user's operating experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 A flow chart of a table extraction method according to an embodiment of the present invention is shown;

[0097] Figure 2 A schematic diagram of a touch display device according to an embodiment of the present invention is shown;

[0098] Figure 3 A specific flow chart of a table extraction method according to an embodiment of the present invention is shown;

[0099] Figure 4 A flowchart of generating a second table corresponding to a first table according to an embodiment of the present invention is shown;

[0100] Figure 5 A schematic diagram showing a first mask pattern according to an embodiment of the present invention is shown;

[0101] Figure 6 A schematic diagram showing a second mask pattern according to an embodiment of the present invention is shown;

[0102] Figure 7 A schematic diagram of a merging rule for merging a plurality of first fold line segments and a plurality of second fold line segments according to an embodiment of the present invention is shown;

[0103] Figure 8 A structural block diagram of a table extraction device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0104] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0105] Embodiment 1

[0106] Reference Figure 1 , shows a flow chart of a table extraction method according to an embodiment of the present invention. The table extraction method can be applied to a touch display device and may specifically include the following steps:

[0107] Step 101: receiving an operation instruction to take a screenshot of the content displayed on the screen of the touch display device.

[0108] In an embodiment of the present invention, the touch display device includes a touch screen and a control module. The control module includes components such as a processor and a memory. The touch screen may be an optical touch screen or an infrared touch screen.

[0109] A user can perform touch operations on the screen of a touch display device (i.e., a touch screen). When a touch operation is received on the screen, a sliding track corresponding to the user's touch operation is displayed on the screen. At this time, the content displayed on the screen is drawn by the user on the touch display device; or, the content displayed on the screen is content received from other terminals and sent, and the content is also manually drawn.

[0110] When the user needs to save the content displayed on the screen, the user can trigger an operation instruction to take a screenshot of the content displayed on the screen, and then receive an operation instruction to take a screenshot of the content displayed on the screen of the touch display device.

[0111] like Figure 2 As shown, the user performs a touch operation on the screen 20 of the touch display device, and the corresponding sliding track forms a first table 21 and characters 22 in the first table 21 , and the first table 21 and characters 22 in the first table 21 are displayed on the screen 20 .

[0112] A save control 23 is also provided on the screen 20 of the touch display device. The save control 23 can be set at the lower right corner of the screen 20. After the user completes drawing the first table 21 and the characters 22 in the first table 21 on the screen 20, if the user needs to save the content drawn on the screen 20, the user can click the save control 23, and a touch operation on the save control 23 displayed on the screen 20 of the touch display device can be received. According to the touch operation of the save control 23, an operation instruction for taking a screenshot of the content displayed on the screen 20 is generated, and the operation instruction is received.

[0113] Of course, the operation instruction is not limited to being generated by touching the save control 23 displayed on the screen 20, but can also be generated by touching an I / O (Input / Output) device connected to the touch display device. The input / output device is a keyboard, mouse, remote control, etc. The user can generate the operation instruction by operating the input / output device.

[0114] Step 102: taking a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen.

[0115] In the embodiment of the present invention, after receiving an operation instruction to take a screenshot of the content displayed on the screen 20 of the touch display device, a screenshot is taken of the content displayed on the screen 20 according to the operation instruction to obtain a screenshot image.

[0116] In actual application, a screenshot is taken of the content displayed on the entire screen 20, rather than just the content displayed on a specific area of ​​the screen 20. Therefore, the screenshot image obtained after the screenshot includes the first table 21 displayed on the screen, and of course, the screenshot image also includes the characters 22 in the first table 21.

[0117] It should be noted that the content displayed on the screen 20 includes multiple first tables, so the screenshot obtained after the screenshot will include multiple first tables, and each first table includes one or more characters, which can be Chinese characters, English characters, numbers, etc.

[0118] Step 103: Detect the screenshot to obtain a second table corresponding to the first table.

[0119] In the embodiment of the present invention, after obtaining the screenshot, the screenshot is detected to obtain a second table corresponding to the first table. The second table is different from the first table in that the first table may contain curve segments, while the second table contains broken line segments obtained by fitting the curve segments.

[0120] When the second table is obtained, table information corresponding to the second table can be obtained accordingly, and the table information includes the number of second tables, the number of rows and columns of each second table, and the first position information of each cell included in each second table.

[0121] Step 104: Identify characters in the screenshot and establish a matching relationship between the second table and the characters.

[0122] In an embodiment of the present invention, a character recognition model is pre-trained using deep learning technology. Specifically, multiple characters are obtained, and the characters are input into an initial recognition model to obtain an output character. According to the difference between the output character and the input character, the parameters in the initial recognition model are corrected. According to the above method, characters are input into the initial recognition model in sequence until the output character is consistent with the input character, and the training process is terminated to obtain the character recognition model.

[0123] After obtaining the screenshot, a character recognition model is used to recognize characters in the screenshot, and each cell in the second table is matched with the character, that is, each cell in the second table is matched with the character, thereby establishing a matching relationship between the second table and the character.

[0124] Step 105: Save the second table, the characters and the matching relationship to obtain a target document.

[0125] In an embodiment of the present invention, after obtaining the second table, the characters, and the matching relationship between the second table and the characters, the second table, the characters, and the matching relationship are saved to obtain the target document. Specifically, the second table, the characters, and the matching relationship can be saved in the touch display device, such as in a memory of the touch display device, or sent to the cloud or other terminals for storage.

[0126] The format of the target document can be word, excel, etc. If the second table, characters and matching relationships are saved in the touch display device, they are saved in a designated directory of the touch display device. If the user needs to view or edit the content previously drawn on the screen 20 again, the user can find the saved target document in the designated directory, open the target document, and edit the target document again.

[0127] It should be noted that when saving the second table, it is actually necessary to save the table information corresponding to the second table, that is, the number of second tables, the number of rows and columns of each second table, and the first position information of each cell included in each second table; after saving the target document, the user only sees the saved document in word or excel format, and cannot directly see the table information corresponding to the second table.

[0128] In an optional implementation manner of the present invention, after step 105, the method further includes:

[0129] Generate a document title according to the storage time of the target document;

[0130] Alternatively, receive a document title input by the user;

[0131] The document title is added to the target document.

[0132] After the second table, characters and matching relationship are saved to obtain the target document, a document title can be added to the target document. The first method is to obtain the saving time of the target document, generate a document title according to the saving time of the target document, and then add the document title to the target document. For example, if the saving time of the target document is 18:00 on December 31, 2019, a document title of 201912311800 is generated according to the saving time of the target document, and the document title of 201912311800 is added to the target document; the second method is to display an input box on the screen 20 after saving the second table, characters and matching relationship to obtain the target document, and the user can enter the document title in the input box, then the document title entered by the user is received, and then the document title is added to the target document. At this time, the document title is a name set by the user, such as a name based on the content of the document title, the usage scenario, etc.

[0133] In an embodiment of the present invention, when the user needs to save the content displayed on the screen, the user can take a screenshot of the content displayed on the screen by triggering an operation instruction to take a screenshot of the content displayed on the screen, and then automatically extract the second table corresponding to the first table in the screenshot and the characters in the second table, and save the second table and the characters as a target document. The user can subsequently edit the content displayed on the screen again based on the target document, thereby improving the user's operating experience.

[0134] Embodiment 2

[0135] Reference Figure 3 , shows a specific flow chart of a table extraction method according to an embodiment of the present invention, which may specifically include the following steps:

[0136] Step 301: receiving an operation instruction to take a screenshot of the content displayed on the screen of the touch display device.

[0137] This step is similar in principle to step 101 in the first embodiment, and will not be described again here.

[0138] Step 302: Take a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen.

[0139] This step is similar in principle to step 102 in the first embodiment, and will not be described again here.

[0140] Step 303: Identify candidate curve segments corresponding to the first table in the screenshot.

[0141] In an embodiment of the present invention, after obtaining a screenshot, candidate curve segments corresponding to a first table in the screenshot are identified, the candidate curve segments including first candidate curve segments arranged along a first direction and second candidate curve segments arranged along a second direction, the first direction may be a row direction, and the second direction may be a column direction.

[0142] Reference Figure 4 , shows a flow chart of generating a second table corresponding to a first table according to an embodiment of the present invention, such as Figure 4 As shown, step 303 may specifically include sub-steps 3031 to 3035:

[0143] Sub-step 3031, processing the screenshot to obtain a binary image;

[0144] Sub-step 3032, in accordance with a first preset order, using a first mask pattern to sequentially traverse the neighboring pixels of each candidate pixel in the binary image row by row;

[0145] Sub-step 3033, determining neighboring pixel points having the same set value as the candidate pixel point as the same type of pixel points, and obtaining a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction;

[0146] Sub-step 3034, using a second mask pattern to sequentially traverse the neighboring pixels of each candidate pixel in the binary image column by column according to a second preset order;

[0147] Sub-step 3035, determining the neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtaining multiple second candidate curve segments corresponding to the first table; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

[0148] In a specific embodiment of the present invention, after obtaining the screenshot, first, the screenshot is grayed to obtain a grayed image; because when the user draws on the screen 20, there are traces only at the drawn position and no traces at the undrawn position, usually the number of pixels at the drawn position on the screen 20 is much smaller than the number of pixels at the undrawn position, therefore, the gray value of each pixel in the grayed image is counted, and the maximum number of gray value occurrences is found, the gray value corresponding to the maximum value is the gray value of the background pixel (the pixel at the undrawn position), and the remaining gray values ​​are the gray values ​​of the foreground pixel (the pixel at the drawing position), the set value of the background pixel is set to 0, and the set value of the foreground pixel is set to 1, thereby obtaining a binary image.

[0149] Next, according to the first preset order, adopt Figure 5 The first mask pattern shown in the figure traverses the neighboring pixel points of each candidate pixel point in the binary image row by row. The first preset order is actually a bottom-up and left-to-right order. The first part 51 in the first mask pattern refers to a candidate pixel point selected in the process of traversing the binary image, and the second part 52 in the first mask pattern represents the neighboring pixel points of the candidate pixel point 51. Figure 5 As shown, one candidate pixel point 51 corresponds to six neighboring pixel points 52, while the pixel point corresponding to the third part 53 in the first mask pattern is not within the search neighborhood range of the candidate pixel point 51 and no processing is performed, that is, the pixel point in the third part 53 will not subsequently perform the judgment of whether the set value is the same.

[0150] The set value of each of the multiple neighborhood pixel points 52 is judged with the set value of the candidate pixel point 51 respectively. If there is a neighborhood pixel point 52 with the same set value as the candidate pixel point 51 among the multiple neighborhood pixel points 52 of the candidate pixel point 51, the neighborhood pixel point 52 with the same set value as the candidate pixel point 51 is determined as a similar pixel point, and the similar pixel points of each candidate pixel point 51 are clustered, while the neighborhood pixel points 52 with different set values ​​from the candidate pixel point 51 are not clustered; when all the neighborhood pixel points 52 of a candidate pixel point 51 are judged, the candidate pixel point 51 is moved to the position of the next pixel point according to the first preset order, and so on, to obtain a plurality of first candidate curve segments corresponding to the first table, and the first candidate curve segments are arranged along the first direction, that is, the first candidate curve segments are arranged along the row direction.

[0151] Accordingly, according to the second preset order, the following is adopted: Figure 6The second mask pattern shown in FIG. 1 traverses the neighboring pixel points of each candidate pixel point in the binary image column by column. The second preset order is actually also a bottom-up, left-to-right order. The first part 61 in the second mask pattern refers to a candidate pixel point selected in the process of traversing the binary image, and the second part 62 in the second mask pattern represents the neighboring pixel points of the candidate pixel point 61. Figure 6 As shown, one candidate pixel point 61 corresponds to six neighboring pixel points 62, while the pixel point corresponding to the third part 63 in the second mask pattern is not within the search neighborhood range of the candidate pixel point 61 and no processing is performed, that is, the pixel point in the third part 63 will not subsequently perform the judgment of whether the set value is the same.

[0152] The set value of each of the multiple neighborhood pixel points 62 is judged with the set value of the candidate pixel point 61 respectively. If there is a neighborhood pixel point 62 with the same set value as the candidate pixel point 61 among the multiple neighborhood pixel points 62 of the candidate pixel point 61, the neighborhood pixel point 62 with the same set value as the candidate pixel point 61 is determined as a similar pixel point, and the similar pixel points of each candidate pixel point 61 are clustered, while the neighborhood pixel points 62 with different set values ​​from the candidate pixel point 61 are not clustered; when all the neighborhood pixel points 62 of a candidate pixel point 61 are judged, the candidate pixel point 61 is moved to the position of the next pixel point according to the second preset order, and so on, to obtain a plurality of second candidate curve segments corresponding to the first table, and the second candidate curve segments are arranged along the second direction, that is, the second candidate curve segments are arranged along the column direction, and the first direction and the second direction are perpendicular.

[0153] It should be noted that the set value of each candidate pixel point (candidate pixel point 51 or candidate pixel point 61) is 1, that is, when traversing the neighborhood pixels of each candidate pixel point in the binary image, only the foreground pixel points with the set value of 1 are taken as candidate pixels, and the background pixel points with the set value of 0 are directly skipped and not processed, that is, it is not judged whether there are similar pixels with the same set value as the background pixel point in the neighborhood pixels of the background pixel point with the set value of 0.

[0154] Step 304: Process the candidate curve segments to generate a second table corresponding to the first table.

[0155] In the embodiment of the present invention, after obtaining the candidate curve segments corresponding to the first table, the candidate curve segments are processed to generate a second table corresponding to the first table. The candidate curve segments include first candidate curve segments arranged along the first direction and second candidate curve segments arranged along the second direction, the first candidate curve segments and the second candidate curve segments are processed respectively to generate first table lines and second table lines, and the second table is generated according to the first table lines and the second table lines.

[0156] When the second table is obtained, the table information corresponding to the second table can be obtained accordingly. The table information includes the number of second tables, the number of rows and columns of each second table and the first position information of each cell included in each second table. The first position information of the cell can be represented by the first position coordinates (u1, v1, u2, v2), (u1, v1) represents the coordinates of the upper left corner of the cell, and (u2, v2) represents the coordinates of the lower right corner of the cell.

[0157] like Figure 4 As shown, step 304 may specifically include sub-steps 3041 to 3043:

[0158] Sub-step 3041, fitting each of the first candidate curve segments and each of the second candidate curve segments respectively to obtain fitted first broken line segments and second broken line segments;

[0159] Sub-step 3042, merging the plurality of first fold line segments and the plurality of second fold line segments respectively according to a first preset condition to obtain corresponding first table lines and second table lines;

[0160] Sub-step 3043: generating a second table corresponding to the first table according to the first table line and the second table line.

[0161] In a specific implementation of the present invention, since the candidate curve segments in the first table manually drawn by the user are usually curves, after obtaining multiple first candidate curve segments and multiple second candidate curve segments, first, each first candidate curve segment is fitted to obtain a first broken line segment after fitting, ensuring that the distance from each point in the first candidate curve segment to the first broken line segment is less than a set threshold, so that the first broken line segment after fitting better meets the requirements, and correspondingly, each second candidate curve segment is fitted to obtain a second broken line segment after fitting, ensuring that the distance from each point in the second candidate curve segment to the second broken line segment is less than a set threshold, so that the second broken line segment after fitting also meets the requirements. Both the first broken line segment and the second broken line segment are broken line segments composed of multiple interconnected straight line segments, and the set threshold is set according to actual conditions.

[0162] Then, since the user may draw the first table manually, the candidate curve segments belonging to the same table line may be drawn as two disconnected curve segments. Therefore, after fitting the first broken line segment and the second broken line segment, multiple first broken line segments are merged according to the first preset condition, that is, the first broken line segments that may belong to the same table line are merged to obtain the first table line, and multiple second broken line segments are merged according to the first preset condition, that is, the second broken line segments that may belong to the same table line are merged to obtain the second table line.

[0163] Finally, after obtaining the first table line and the second table line, a second table corresponding to the first table is generated according to the first table line and the second table line. The first table line is a table line arranged along the row direction, and the second table line is a table line arranged along the column direction.

[0164] In an optional implementation manner of the present invention, before sub-step 3041, the method further includes: filtering out the first candidate curve segment and the second candidate curve segment whose lengths are less than the first preset length.

[0165] In actual application, after obtaining multiple first candidate curve segments and multiple second candidate curve segments, the length of each first candidate curve segment and each second candidate curve segment can be obtained, and the first candidate curve segments and the second candidate curve segments whose lengths are less than the first preset length can be screened out, that is, the curve segments that are too short are deleted. This does not affect the shape of the second table generated subsequently, and can reduce the number of candidate curve segments when fitting the candidate curve segments subsequently, thereby simplifying the operation process.

[0166] In an optional implementation of the present invention, sub-step 3042 may include step A1 and step A2:

[0167] Step A1: for any two first folded line segments, if the endpoint of a first straight line segment in a first candidate folded line segment is located within a first preset range where the endpoint of a second straight line segment in a first folded line segment to be merged is located, merge the first candidate folded line segment and the first folded line segment to be merged;

[0168] Step A2: for any two second broken line segments, if the endpoint of the first straight line segment in the second candidate broken line segment is located within the first preset range where the endpoint of the second straight line segment in the second broken line segment to be merged is located, merge the second candidate broken line segment and the second broken line segment to be merged.

[0169] like Figure 7 As shown, 71 represents the first candidate broken line segment among any two first broken line segments (only the first straight line segment among the first candidate broken line segments is shown), or the second candidate broken line segment among any two second broken line segments (only the first straight line segment among the second candidate broken line segments is shown); 72 represents the first broken line segment to be merged among any two first broken line segments or the second broken line segment to be merged among any two second broken line segments.

[0170] Taking 71 as an example to represent the first candidate polyline segment of any two first polyline segments and 72 as the first polyline segment to be merged of any two first polyline segments, the principle of merging the first candidate polyline segment and the first polyline segment to be merged is explained:

[0171] The first candidate broken line segment 71 includes a first straight line segment, and the endpoint of the first straight line segment is Q. The first broken line segment 72 to be merged includes a first straight line segment 721 and a second straight line segment 722, and the endpoint of the second straight line segment 722 is P. If the endpoint Q of the first straight line segment in the first candidate broken line segment 71 is located within the first preset range 73 where the endpoint P of the second straight line segment 722 in the first broken line segment 72 to be merged is located, it is determined that the first candidate broken line segment 71 and the first broken line segment 72 to be merged may belong to the same table line, and the first candidate broken line segment 71 and the first broken line segment 72 to be merged are merged.

[0172] Correspondingly, for any two second broken line segments, the judgment can be made in the above manner. If the endpoint of the first straight line segment in the second candidate broken line segment is located within the first preset range where the endpoint of the second straight line segment in the second broken line segment to be merged is located, it is determined that the second candidate broken line segment and the second broken line segment to be merged may belong to the same table line, and the second candidate broken line segment and the second broken line segment to be merged are merged.

[0173] The first preset range 73 can be determined by a first preset angle, a second preset angle, and a second preset length. When the first angle α is smaller than the first preset angle, the second angle β is smaller than the second preset angle, and the first length d is smaller than the second preset length, the endpoint Q of the first straight line segment is located within the first preset range where the endpoint P of the second straight line segment 722 is located. The first preset angle, the second preset angle, and the second preset length can be set according to actual conditions.

[0174] Among them, 74 represents the first line segment formed between the endpoint Q of the first straight line segment and the endpoint P of the second straight line segment 722, 75 represents the perpendicular line segment between the endpoint Q of the first straight line segment and the extension line of the second straight line segment 722, the first angle α is the angle between the first line segment 74 and the extension line of the second straight line segment 722, the second angle β is the angle between the extension line of the first straight line segment and the extension line of the second straight line segment 722, and the first length d is the length of the perpendicular line segment 75 between the endpoint Q of the first straight line segment and the extension line of the second straight line segment.

[0175] In an optional implementation manner of the present invention, sub-step 3043 may include step B1, step B2, step B3 and step B4:

[0176] Step B1, sorting a plurality of the second table lines according to a third preset order;

[0177] Step B2, selecting a second table line that meets a second preset condition as a first boundary line from the second table lines according to the sorting result; the second preset condition is that there are two first table lines within a second preset range of two endpoints of the second table line, and the length difference between the two first table lines is less than a third preset length;

[0178] Step B3, selecting a second table line that meets a third preset condition from the remaining second table lines as a second boundary line, and determining the two first table lines as a third boundary line and a fourth boundary line; the third preset condition is that the two endpoints of the second table line are respectively located within a third preset range where the first endpoints of the two first table lines are located;

[0179] Step B4, screening the remaining first table lines and second table lines according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain a table middle line to generate a second table.

[0180] First, after obtaining the first table line and the second table line, the plurality of second table lines are sorted from left to right, that is, the third preset order is the order from left to right. Specifically, the ordinate of each second table line can be obtained, and the ordinates are sorted from small to large, so that the plurality of second table lines can be sorted from left to right.

[0181] After sorting the plurality of second table lines from left to right, starting from the leftmost, it is judged whether each second table line meets the second preset condition in turn. If there are two first table lines X1 and X2 within the second preset range of the two end points of the second table line Y1, and the length difference between the two first table lines X1 and X2 is less than the third preset length, the second table line Y1 is determined to be the first boundary line, i.e., the left boundary; if the second table line does not meet the second preset condition, i.e., there are no two first table lines within the second preset range of the two end points of the second table line, and / or the length difference between the two first table lines is not less than the third preset length, it is judged whether the next second table line meets the second preset condition according to the sorting result. The second preset range and the third preset length can be set according to actual conditions.

[0182] After determining the first boundary line Y1, from the remaining second table lines, determine in turn whether the two endpoints of each second table line are respectively located within the third preset range where the first endpoints of the two first table lines X1 and X2 are located, the first endpoint is the right endpoint of the first table lines X1 and X2, if the two endpoints of the second table line are located within the third preset range where the first endpoints of the two first table lines X1 and X2 are located, then determine that the second table line meets the third preset condition, and use it as the second boundary line Y2, and determine the two first table lines X1 and X2 as the third boundary line and the fourth boundary line, that is, the second boundary line Y2 is the right boundary line, the third boundary line X1 is the upper boundary line, and the fourth boundary line X2 is the lower boundary line, thereby obtaining the upper, lower, left and right four boundary lines of the second table, and the second boundary line Y2 and the first boundary line Y1 are not the same second table line. The third preset range can be set according to actual conditions.

[0183] Finally, the remaining first table lines and second table lines are screened according to the first boundary line Y1 , the second boundary line Y2 , the third boundary line X1 and the fourth boundary line X2 to obtain the table middle line, so as to generate a second table.

[0184] When the screenshot includes multiple first tables, in the above manner, steps B2 to B4 are executed to re-determine the left boundary, right boundary, upper boundary and lower boundary of the remaining first table lines and second table lines until there are no remaining first table lines and second table lines.

[0185] Specifically, step B4 may include step B41 and step B42:

[0186] Step B41, selecting a first table line that meets a fourth preset condition from the remaining first table lines as a table middle line arranged along the first direction;

[0187] Step B42, selecting a second table line that meets the fifth preset condition from the remaining second table lines as a table middle line arranged along the second direction;

[0188] Among them, the fourth preset condition is that the distances between the two endpoints of the first table line and the first boundary line and the second boundary line are both less than the first preset distance; the fifth preset condition is that the distances between the two endpoints of the second table line and the third boundary line and the fourth boundary line are both less than the second preset distance.

[0189] After determining the first boundary line Y1, the second boundary line Y2, the third boundary line X1 and the fourth boundary line X2, it is determined whether each of the remaining first table lines meets the fourth preset condition, that is, it is determined whether the distances between the two endpoints of each of the remaining first table lines and the first boundary line Y1 and the second boundary line Y2 are both less than the first preset distance. If so, the first table line is the middle line of the table arranged along the first direction; correspondingly, it is determined whether each of the remaining second table lines meets the fifth preset condition, that is, it is determined whether the distances between the two endpoints of each of the remaining second table lines and the third boundary line X1 and the fourth boundary line X2 are both less than the second preset distance. If so, the second table line is the middle line of the table arranged along the second direction. The first preset distance and the second preset distance may be equal or different, and the first preset distance and the second preset distance may be set according to actual conditions.

[0190] Step 305: Identify characters in the screenshot.

[0191] In the embodiment of the present invention, after the screenshot is obtained, a character recognition model is used to recognize characters in the screenshot.

[0192] Step 306: Obtain first position information of each cell in the second table.

[0193] In an embodiment of the present invention, after obtaining the second table, the table information corresponding to the second table can be obtained accordingly, and the table information includes the number of second tables, the number of rows and columns of each second table, and the first position information of each cell included in each second table, and the first position information of each cell in the second table is obtained.

[0194] Step 307: Obtain second position information of each of the characters.

[0195] In the embodiment of the present invention, when each character is recognized, the second position information of each character can be obtained accordingly. The second position information of each character can be obtained, and the second position information can be represented by coordinates (a1, b1).

[0196] Step 308: Establish a matching relationship between the second table and the character according to the first position information and the second position information.

[0197] In an embodiment of the present invention, the first position information of each cell in the second table and the second position information of the character are matched. The first position information of the cell can be represented by the first position coordinates (u1, v1, u2, v2), (u1, v1) represents the coordinates of the upper left corner of the cell, (u2, v2) represents the coordinates of the lower right corner of the cell, and the second position information of the character can be represented by the coordinates (a1, b1). If the horizontal coordinate a1 of the character is between u1 and u2, and the vertical coordinate b1 of the character is between v1 and v2, it is determined that the character is located in the cell, and then a matching relationship between the cell and the character in the second table is established.

[0198] Step 309: Save the second table, the characters and the matching relationship to obtain a target document.

[0199] This step is similar in principle to step 105 in the first embodiment, and will not be described again here.

[0200] In an embodiment of the present invention, when the user needs to save the content displayed on the screen, the user can take a screenshot of the content displayed on the screen by triggering an operation instruction to take a screenshot of the content displayed on the screen, and then automatically extract the candidate curve segments corresponding to the first table in the screenshot image, process the candidate curve segments to obtain a second table, and then identify the characters in the screenshot image, establish a matching relationship between the second table and the characters based on the first position information of each cell in the second table and the second position information of the characters, and save the second table, the characters and the matching relationship as a target document. The user can subsequently edit the content displayed on the screen again based on the target document, thereby improving the user's operating experience.

[0201] Embodiment 3

[0202] Reference Figure 8 , shows a structural block diagram of a table extraction device according to an embodiment of the present invention.

[0203] The table extraction device 800 of the embodiment of the present invention is applied to a touch display device, and the table extraction device 800 includes:

[0204] An operation instruction receiving module 801 is configured to receive an operation instruction to take a screenshot of the content displayed on the screen of the touch display device;

[0205] The screenshot module 802 is configured to take a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen;

[0206] A table detection module 803 is configured to detect the screenshot image to obtain a second table corresponding to the first table;

[0207] A character recognition module 804 is configured to recognize characters in the screenshot and establish a matching relationship between the second table and the characters;

[0208] The saving module 805 is configured to save the second table, the characters and the matching relationship to obtain a target document.

[0209] Optionally, the table detection module 803 includes:

[0210] a candidate curve segment identification submodule, configured to identify candidate curve segments corresponding to the first table in the screenshot;

[0211] The second table generating submodule is configured to process the candidate curve segments to generate a second table corresponding to the first table.

[0212] Optionally, the candidate curve segment identification submodule includes:

[0213] A picture processing unit, configured to process the screenshot picture to obtain a binary image;

[0214] A first traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image row by row in sequence using a first mask pattern according to a first preset order;

[0215] A first candidate curve segment determination unit is configured to determine neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtain a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction;

[0216] A second traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image column by column in sequence using a second mask pattern according to a second preset order;

[0217] The second candidate curve segment determination unit is configured to determine neighboring pixel points having the same set value as the candidate pixel point as the same type of pixel points, and obtain multiple second candidate curve segments corresponding to the first table; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

[0218] Optionally, the second table generating submodule includes:

[0219] a candidate curve segment fitting unit, configured to fit each of the first candidate curve segments and each of the second candidate curve segments respectively to obtain fitted first broken line segments and second broken line segments;

[0220] A broken line segment merging unit, configured to merge the plurality of the first broken line segments and the plurality of the second broken line segments according to a first preset condition, respectively, to obtain corresponding first table lines and second table lines;

[0221] The second table generating unit is configured to generate a second table corresponding to the first table according to the first table line and the second table line.

[0222] Optionally, the table extraction device 800 further includes:

[0223] The candidate curve segment screening unit is configured to screen out the first candidate curve segment and the second candidate curve segment whose lengths are less than a first preset length.

[0224] Optionally, the broken line segment merging unit includes:

[0225] A first merging subunit is configured to merge any two first polyline segments, if the endpoint of a first straight line segment in the first candidate polyline segment is located within a first preset range where the endpoint of a second straight line segment in the first polyline segment to be merged is located, the first candidate polyline segment and the first polyline segment to be merged are merged;

[0226] A second merging subunit is configured to merge any two second polyline segments, if the endpoint of the first straight line segment in the second candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the second polyline segment to be merged is located, the second candidate polyline segment and the second polyline segment to be merged are merged;

[0227] The first preset range is determined by a first preset angle, a second preset angle, and a second preset length; when the first angle is smaller than the first preset angle, the second angle is smaller than the second preset angle, and the first length is smaller than the second preset length, the endpoint of the first straight line segment is located within the first preset range where the endpoint of the second straight line segment is located;

[0228] A first line segment is formed between the endpoint of the first straight line segment and the endpoint of the second straight line segment, the first angle is the angle between the first line segment and the extension line of the second straight line segment, the second angle is the angle between the extension line of the first straight line segment and the extension line of the second straight line segment, and the first length is the length of the perpendicular segment between the endpoint of the first straight line segment and the extension line of the second straight line segment.

[0229] Optionally, the second table generating unit includes:

[0230] A second table line sorting subunit is configured to sort the plurality of second table lines according to a third preset order;

[0231] The first boundary line determination subunit is configured to select, from the second table lines according to the sorting result, a second table line that meets a second preset condition as the first boundary line; the second preset condition is that there are two first table lines within a second preset range of two end points of the second table line, and the length difference between the two first table lines is less than a third preset length;

[0232] The second boundary line determination subunit is configured to select a second table line that satisfies a third preset condition from the remaining second table lines as the second boundary line, and determine the two first table lines as the third boundary line and the fourth boundary line; the third preset condition is that the two end points of the second table line are respectively located within a third preset range where the first end points of the two first table lines are located;

[0233] The second table generating subunit is configured to screen the remaining first table lines and second table lines according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain a table middle line to generate a second table.

[0234] Optionally, the second table generating subunit is specifically configured as follows:

[0235] Selecting, from the remaining first table lines, a first table line that meets a fourth preset condition as a table middle line arranged along the first direction;

[0236] From the remaining second table lines, selecting a second table line that meets the fifth preset condition as a table middle line arranged along the second direction;

[0237] Among them, the fourth preset condition is that the distances between the two endpoints of the first table line and the first boundary line and the second boundary line are both less than the first preset distance; the fifth preset condition is that the distances between the two endpoints of the second table line and the third boundary line and the fourth boundary line are both less than the second preset distance.

[0238] Optionally, the character recognition module 804 includes:

[0239] A first position information acquisition submodule, configured to acquire first position information of each cell in the second table;

[0240] A second position information acquisition submodule is configured to acquire the second position information of each of the characters;

[0241] The matching relationship establishing submodule is configured to establish a matching relationship between the second table and the character according to the first position information and the second position information.

[0242] Optionally, the table extraction device 800 further includes:

[0243] A first title generation module is configured to generate a document title according to the storage time of the target document;

[0244] Alternatively, the second title generation module is configured to receive a document title input by a user;

[0245] The title adding module is configured to add the document title to the target document.

[0246] In an embodiment of the present invention, when the user needs to save the content displayed on the screen, the user can take a screenshot of the content displayed on the screen by triggering an operation instruction to take a screenshot of the content displayed on the screen, and then automatically extract the second table corresponding to the first table in the screenshot and the characters in the second table, and save the second table and the characters as a target document. The user can subsequently edit the content displayed on the screen again based on the target document, thereby improving the user's operating experience.

[0247] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0248] Correspondingly, an embodiment of the present invention also provides a touch display device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned table extraction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0249] The embodiment of the present invention further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned table extraction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0250] For the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0251] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0252] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0253] The table extraction method, device and touch display device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A table extraction method, characterized in that: Applied to a touch display device, the method comprises: Receiving an operation instruction to take a screenshot of the content displayed on the screen of the touch display device; Taking a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen; Detecting the screenshot to obtain a second table corresponding to the first table; Recognize characters in the screenshot, and establish a matching relationship between the second table and the characters; saving the second table, the characters and the matching relationship to obtain a target document; The step of detecting the screenshot to obtain the second table corresponding to the first table includes: identifying candidate curve segments corresponding to the first table in the screenshot, processing the candidate curve segments, and generating the second table corresponding to the first table; The step of identifying candidate curve segments corresponding to the first table in the screenshot includes: Processing the screenshot to obtain a binary image; According to a first preset order, the first mask pattern is used to sequentially traverse the neighboring pixel points of each candidate pixel point in the binary image row by row; Determine neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtain a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction; According to a second preset order, the second mask pattern is used to sequentially traverse the neighboring pixel points of each candidate pixel point in the binary image column by column; Neighboring pixels having the same set value as the candidate pixel are determined as similar pixels, and a plurality of second candidate curve segments corresponding to the first table are obtained; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

2. The method according to claim 1, characterized in that The step of processing the candidate curve segments to generate a second table corresponding to the first table includes: Fitting each of the first candidate curve segments and each of the second candidate curve segments respectively to obtain fitted first broken line segments and second broken line segments; Merging the plurality of first fold line segments and the plurality of second fold line segments according to a first preset condition to obtain corresponding first table lines and second table lines; A second table corresponding to the first table is generated according to the first table line and the second table line.

3. The method according to claim 2, characterized in that Before the step of fitting each of the first candidate curve segments and each of the second candidate curve segments respectively, the method further includes: The first candidate curve segment and the second candidate curve segment whose length is less than the first preset length are screened out.

4. The method according to claim 2, characterized in that: The step of merging the plurality of first fold line segments and the plurality of second fold line segments according to the first preset condition to obtain corresponding first table lines and second table lines comprises: For any two first polyline segments, if the endpoint of the first straight line segment in the first candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the first polyline segment to be merged is located, merge the first candidate polyline segment and the first polyline segment to be merged; For any two second polyline segments, if the endpoint of the first straight line segment in the second candidate polyline segment is located within a first preset range where the endpoint of the second straight line segment in the second polyline segment to be merged is located, merge the second candidate polyline segment and the second polyline segment to be merged; The first preset range is determined by a first preset angle, a second preset angle, and a second preset length; when the first angle is smaller than the first preset angle, the second angle is smaller than the second preset angle, and the first length is smaller than the second preset length, the endpoint of the first straight line segment is located within the first preset range where the endpoint of the second straight line segment is located; A first line segment is formed between the endpoint of the first straight line segment and the endpoint of the second straight line segment, the first angle is the angle between the first line segment and the extension line of the second straight line segment, the second angle is the angle between the extension line of the first straight line segment and the extension line of the second straight line segment, and the first length is the length of the perpendicular segment between the endpoint of the first straight line segment and the extension line of the second straight line segment.

5. The method according to claim 2, characterized in that: The step of generating a second table corresponding to the first table according to the first table line and the second table line includes: sorting the plurality of second table lines according to a third preset order; According to the sorting result, a second table line satisfying a second preset condition is selected from the second table lines as the first boundary line; the second preset condition is that there are two first table lines within a second preset range of two end points of the second table line, and the length difference between the two first table lines is less than a third preset length; From the remaining second table lines, a second table line that meets a third preset condition is selected as the second boundary line, and the two first table lines are determined as the third boundary line and the fourth boundary line; the third preset condition is that the two endpoints of the second table line are respectively located within a third preset range where the first endpoints of the two first table lines are located; The remaining first table lines and second table lines are screened according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain a table middle line to generate a second table.

6. The method according to claim 5, characterized in that The step of screening the remaining first table lines and second table lines according to the first boundary line, the second boundary line, the third boundary line and the fourth boundary line to obtain the table middle line includes: Selecting, from the remaining first table lines, a first table line that meets a fourth preset condition as a table middle line arranged along the first direction; From the remaining second table lines, selecting a second table line that meets the fifth preset condition as a table middle line arranged along the second direction; Among them, the fourth preset condition is that the distances between the two endpoints of the first table line and the first boundary line and the second boundary line are both less than the first preset distance; the fifth preset condition is that the distances between the two endpoints of the second table line and the third boundary line and the fourth boundary line are both less than the second preset distance.

7. The method according to claim 1, characterized in that The step of establishing a matching relationship between the second table and the characters comprises: Obtaining first position information of each cell in the second table; Obtaining second position information of each of the characters; A matching relationship between the second table and the character is established according to the first position information and the second position information.

8. The method according to any one of claims 1 to 7, characterized in that After the step of saving the second table, the characters and the matching relationship to obtain the target document, the method further includes: Generate a document title according to the storage time of the target document; Alternatively, receive a document title input by the user; The document title is added to the target document.

9. A table extraction device, characterized in that: Applied in a touch display device, the table extraction device comprises: An operation instruction receiving module, configured to receive an operation instruction to take a screenshot of the content displayed on the screen of the touch display device; A screenshot module, configured to take a screenshot of the content displayed on the screen according to the operation instruction to obtain a screenshot image; the screenshot image includes a first table displayed on the screen; a table detection module, configured to detect the screenshot image to obtain a second table corresponding to the first table; a character recognition module, configured to recognize characters in the screenshot image and establish a matching relationship between the second table and the characters; A saving module, configured to save the second table, the characters and the matching relationship to obtain a target document; The table detection module includes: a candidate curve segment identification submodule configured to identify candidate curve segments corresponding to the first table in the screenshot; a second table generation submodule configured to process the candidate curve segments to generate a second table corresponding to the first table; The candidate curve segment identification submodule includes: A picture processing unit is configured to process the screenshot picture to obtain a binary image; A first traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image row by row in sequence using a first mask pattern according to a first preset order; A first candidate curve segment determination unit is configured to determine neighboring pixel points with the same set value as the candidate pixel point as the same type of pixel points, and obtain a plurality of first candidate curve segments corresponding to the first table; the first candidate curve segments are arranged along a first direction; A second traversal unit is configured to traverse the neighboring pixel points of each candidate pixel point in the binary image column by column in sequence using a second mask pattern according to a second preset order; The second candidate curve segment determination unit is configured to determine neighboring pixel points having the same set value as the candidate pixel point as the same type of pixel points, and obtain multiple second candidate curve segments corresponding to the first table; the second candidate curve segments are arranged along a second direction, and the first direction is perpendicular to the second direction.

10. A touch display device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the table extraction method according to any one of claims 1 to 8 when executed by the processor.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the table extraction method according to any one of claims 1 to 8 are implemented.

Images